Category Archives: Analysis Techniques

Articles on how to be an analyst

An Interesting Data Point on Sea Org Headcount and Revenue

Sea Org executives back in the good ol' (or at least more numerous) days

Sea Org executives back in the good ol’ (or at least more numerous) days. Why do their hats look just a bit too large for their heads, in yet another parallel to the North Korean military?

In a video posted Sunday on Tony Ortega’s site, recently escaped Sea Org member Jillian Schlesinger talked about her time in the cult.  Jillian related several data points that are of interest, including David Miscavige’s claim that RTC is reaping $75 million per year in revenue from Flag in Clearwater.

Here, we look at a few of the things Jillian said, and we look at what this might mean for the cult.  Throughout, I share some analytical techniques I’m using to evaluate some of the things Jillian said when they either confirm or contradict my current best guesses. These thoughts may help you to do a better job in your own analysis.

Perhaps the most interesting quote from Jillian’s video was something Scientology leader David Miscavige said to the assembled Sea Org troops in LA:

There was an analysis done some months ago, probably about six months ago now. And he [David Miscavige] had revealed that in Florida, there’s about 2,000 Sea Org members, and they send up $1.5 million a week to him. Then, looking in all of LA, there’s about 2,000 Sea Org members, and they were not sending up that much to him every week. So, he wanted a huge rearrangement of staff, and then to make a, in-house, out-of-Sea-Org-member, construction unit, so that we could save money doing it ourselves, even though we have no training in construction, and then we don’t have to hire anyone. And then I was doing that for about the last six months.

How Many Sea Org Are There, Really?

This quote from COB is interesting in two respects.  First, of course, Miscavige is saying that there are at least 4,000 Sea Org members in the US, approximately evenly distributed on both coasts.  And he’s claiming that he’s getting $75 million per year from Flag and a lesser amount from the Western US.  My estimates are that the real number from Flag is closer to $30 million per year these days, perhaps a bit less, and that they’re probably getting $10 million from the higher level orgs on the West Coast (it’s not an international destination for people coming in for advanced training like Flag is).  And based on some correspondence late last year with a couple Anons in the Clearwater area, it appears that some of the cult-owned apartment buildings used to house Flag workers are sitting empty.

So how does a careful analyst think about the disparity between previous estimates and the new data point?  This is one of the trickier analytical traps: it’s dangerous to discount new information when it doesn’t fit your biases, but it’s equally dangerous to believe implicitly that a newer data point means your prior thinking is wrong.  If the new information is correct, it would mean that my estimates of about $180 million in cult revenue in 2013 were low by an integer multiple — the cult could have been raking in $400 or $500 million a year; that’s a major error.  Intellectual honesty, essential for a good analyst, means we have to look at a surprising new data point with a willingness to consider honestly the possibility that we’re way off in our numbers.

The first thing to do in teasing apart this information is to consider the credibility of the source.  It’s probably reasonable to believe that someone emerging from the Sea Org only a couple weeks ago and re-adjusting to reality is not going to have the spare time to fabricate a story like this out of whole cloth. It is thus reasonable to believe she’s recounting accurately what she heard.  The question then becomes whether Miscavige’s numbers are credible.

First Cross-Check: Direct Observation

In looking at the claim of 4,000 Sea Org members, it’s important to cross-check this against other data that we have.  In evaluating the number of Sea Org members in LA, let’s start with easily observable data points.  One such is the level of foot traffic at L. Ron Hubbard Way in LA.  A recent afternoon stroll down the sacred street showed only two people (one a security guard), in front of the building, and two RPF girls on the back side on Catalina St.  No Sea Org was visible.  The parking lot, which holds approximately 200 cars (I counted from the aerial Google Maps view) was only slightly more than half full; even assuming that few Sea Org have cars, that’s a pretty light occupancy load.  These observations are enough to raise suspicions about Miscavige’s credibility, but they’re not conclusive: given the tight constraints on their work schedules, it’s reasonable to expect that one wouldn’t see too many of the Sea Org types out on the sidewalk during nose-to-the-grindstone hours.

Second Cross-Check: Independent Rules of Thumb

The next step is to cross check the claim against another independent metric, one that has nothing to do with the first set of observations.  In other words, you want to look at an independent measure.  It’s often useful to consider real estate square foot per staff member.  I would estimate that the 11 floors (+/-) of the original Big Blue building total about 125,000 square feet, and the rest of the complex is similar sized — call it 250,000 feet total.  While I haven’t done a complete count of the rest of the cult properties in the area, let’s assume another 100,000 square feet.  Let’s then assume that the hypothetical 2,000 Sea Org members are joined by about 1,000 other staff.

If you do the math, that implies there are about 120 square feet of facilities per person, slightly larger than a 10 by 10 foot office.  But that’s quite a bit lower than conventional estimates of office space per capita, which is currently typically about 250 square feet per employee. That’s including conference room facilities, restrooms, and all sorts of other stuff.  Incidentally, various commercial real estate trade groups suggest this number is shrinking rapidly due to “hoteling” (unassigned cubicles shared by multiple workers) due to the rise of telecommuting, a trend that the cult is unlikely to embrace.  I thus am sticking with the 250 square foot number for the foreseeable future even if the overall trend in real estate is expected to drop sharply.

It’s also worth noting that within the original Big Blue building, the floor plate of a 1930s edifice is substantially less efficient than a modern skyscraper, so the percentage of each floor plate devoted to “core” functions like elevators, plumbing, janitorial, restrooms, etc. is a much higher percentage of the space on a given floor in a building of that age than in something more modern. By the way, that explains why the Empire State Building in New York is a great landmark but a terrible office building — the ratio of rentable space on each floor is too low for it to compete against modern buildings.  So in the case of the Big Blue complex, the 250 square foot per person number could actually be low when you measure the gross square footage of the complex (that’s what you measure when you measure the outside dimensions of the building).

Note that in these estimates, we’re not subtracting out square footage for delivering services, places where employees wouldn’t normally be stationed. So we’re not deducting square footage for course rooms at AOLA, for the L. Ron Hubbard life exhibit, for the Psychiatry: Industry of Death exhibit, etc. Again, we’re being “conservative” (using assumptions that are least favorable to our argument).

Using the 250 square feet per employee and dividing that into total square feet, we get about 1,300 employees in Hollywood.  Adjusting for regular staff versus Sea Org, we get to about 600-800 Sea Org in Hollywood.  So Miscavige is lying by a factor of about 2.5x to 3.5x what we believe is observable.  That, too, is consistent (perhaps even on the low side) of the “Miscavige Reality Distortion Ratio” that we’ve often seen from the cult’s “fearless leader.”

And The Same General Principle Holds True in Clearwater…

To cross-check the 2,000 claimed Sea Org in Clearwater, we’ll use a similar process.  Those of you who are denizens of the greater Tampa Bay area and who have better visibility into the real estate holdings of the cult at Flag could help list and then nail down the use of the various buildings, to help me develop an understanding of how much space is customer-related (i.e., non-office), how much is office-related, and how much is guest accommodations.  Yes, that’s a pretty subtle, but definitely impassioned plea for any hard data points that you have.

The diversity of real estate holdings in Clearwater makes the process of getting credible staff counts a bit harder, though it is still ultimately possible to get accurate numbers.  Until we’re able to list all the properties and figure out how much square footage is in each, then categorize them as office, hotel or customer spaces, we’ll try to use metrics other than the total square footage of all the downtown Clearwater buildings.  

At this point, the most useful observation came from an e-mail conversation I had with a former member and current Anon in the area, who sent a preliminary listing of properties including apartments, and who pointed out that a number of the buildings are actually empty at this point.  While I can’t find the exact notes at the moment (ah, the joys of restarting a blog after a hiatus), I am going to guess that staff levels are substantially below peak levels, with perhaps less than half the cult-owned units occupied by staff. Incidentally, it’s entirely possible that the cult over-bought apartment space a few years back, above even staffing levels at the time, so it’s not wise to try to estimate current staffing based on the number of total units.

Perhaps the best check of staff size at Flag will now boil down to the capacity of the staff dining room in the basement of the Super Power building.  The total permitted floor space of the building is 377,000 on eight floors (including the basement), so we’ll assume a floor plate of 50,000 square feet. I have some feelers out for more data about the layout of the basement, but if you subtract out machinery spaces, storage, etc., then it is reasonably likely that no more than half of the basement floor plate will be devoted to the staff dining room. Typically, in a restaurant, about 40% to 50% of the square footage is devoted to the kitchen, and the rest to the dining room. One rule of thumb suggests allocating about 20 square feet per customer. If we thus assume that the total dining space is 25,000 square feet, then a dining room of 12,500 square feet will hold about 600 staff.  Then if we further assume that they have two meal services per meal (easily done given the staff’s regimented lives), then we could believe there are about 1,200 staff at Flag.  But I would not expect hotel staff to have to eat in the staff dining room, so we could add another 300 staff to that number and get about 1,500 total staff in Clearwater.  That cross-checks reasonably well with a quick back-of-the-envelope estimate of the capacity of the occupied apartments.  And again, assuming that at most 2/3 of staff in Clearwater are Sea Org, then we get to about 800 Sea Org at Flag.

The Other Guys are Sending Me $75 Million a Year; What’s Wrong With You Morons?

The second part of Miscavige’s assertion is that he’s pulling $75 million out of Flag per year.  The quote is worded to say that “they send $1.5 million a week up to him.” In other words, that’s the contribution to Int Management and thus to the Sea Org reserves.  Let’s assume that Flag is sending about 75% of their revenue uplines. That implies that the cult is pulling in $2 million a week in gross revenue, about $100 million per year.  How credible is this?

First Cross-Check: No Decline in Flag Revenues in Ten Years? Seriously?

There are a couple of ways to cross-check this.  First, there’s a common sense metric.  Mat Pesch once commented, either in an article on Mike Rinder’s blog, Marty Rathbun’s blog or on Tony Ortega’s site, that when he was in ten years ago, Flag was pulling in about $2 million a week.  Mat was a finance guy at Flag and has been credible in his description of life in the cult, so we’ll believe that the numbers he gave are accurate.  Miscavige is thus claiming that revenue has remained consistent with the decade-ago level. Given all that has happened in the cult, this is ludicrous on its face.  In other words, the bullshit alarm is ringing loudly.  But simply sounding the bullshit alarm isn’t sufficient; we need to look at a couple of other metrics to try and understand what’s really going on in order to demolish this claim convincingly.

Second Cross-Check: Flag Revenue Per Potential Customer

Second, an upper bound of Flag revenue could come from an estimate of per-capita spending from each active member.  My current estimate is that there are 20,000 active public Scientologists.  I believe about 4,000 of those are from countries with relatively low per capita income (Russia, Taiwan, etc.) where travel to Flag on any sort of regular basis is not likely to be affordable. As a result, I believe the total market for Flag services is about 16,000 active cult members.

But it’s important to realize that likely overstates the actual addressable market for Flag services.  An addressable market is one with proven potential, not just the theoretical upper maximum.  The total market for cars is everyone who currently owns a car, plus the number of people who will get a driver’s license, minus those who will be too old to drive further. But the real addressable market for cars are some percentage of those whose leased cars whose leases are up, the number of people whose cars will be totaled in accidents, plus other populations — a far cry from the total number of drivers.

In considering the addressable market of Flag customers, I’m intrigued by the recent interviews I’ve had with ex members.  Consistently, these folks said that they had done everything possible to avoid going to Flag, since they were quite clear that they would be hit up endlessly for donations, and that they would be trapped there for far longer than they intended to stay.  They all pointed out that the pressure to donate when one stayed at Flag had consistently grown worse in their often decades-long experience in the Church.  None had been to Flag in the last ten years before they walked away from the cult.  In other words, they had all managed to avoid the clutches of the FSO reg’s.

Additionally, I believe Europeans are less likely to jump on a plane to come to Flag, partly because of the cost and hassles of much longer flights, but also because of the declasse nature of Clearwater — why go there when you can get many services at a stately English manor home (Saint Hill)?

It’s probably also necessary to cut out second generation members in their early 20s who are on their own, who don’t have the economic firepower to afford two weeks at the Fort Harrison Hotel and all the services.

Finally, one must estimate one of the most productive markets for Flag services: the customers “on the level” (i.e., doing OT VII), who are required to come in for sec checks every six months.  At this point, there are probably less than 500 Scientologists actually on the level.

I thus believe that the actual addressable market for Flag services is no more than 7,000 people (excluding OT VII sec checkers). $100 million divided by 700 people gets you to about $15,000 per person per year, for every single person in the addressable market in every single year, whether or not all of those potential customers actually do come to Flag.  That seems extremely high, though it is not inconceivable.

Importantly, it may have been possible to get $15,000 per Flag visit out of the average Flag visitor in years past if they were doing multiple intensives with Class XII auditors at $1,000 per hour.  But that is probably more the exception than the rule these days: you have to take into effect the effects of the Golden Age of Tech Phase II (GAT 2) announcement.  Remember that GAT 2 is requiring many people to redo lower-level course work, and that the promise of GAT 2 is that you’ll get through all that low-level nonsense so much faster than ever before.  We’ll ignore the opportunity for assorted (and obvious) hilarity when contemplating the joy people will feel from screwing up their brains learning Study Tech in the “Student Hat” course for the fourth time.  What’s important is that the per-hour revenue for getting people to come to Flag to do these low-level course is likely to be significantly smaller than the per-hour revenue from Class XII auditing intensives in the past.  And that suggests strongly that the Flag business is off from levels a decade ago, where the cult had a much larger addressable market (perhaps 40,000 total public worldwide in 2000 versus the current 20,000, and almost all from high-GDP economies back then).  The mix shift away from high-margin services and thus much higher per-hour revenue to lower margin low-level services must inevitably mean that revenue is down.

It’s possible that total Flag revenue is as high as $50 million, a little higher than I had previously been estimating.  I had estimated that all services-related revenue including Flag was about $60 million, but the revenue from the Orgs may actually turn out, based on recent document dumps from Mike Rinder, to be less than the $20 million I had estimated.

Third Cross-Check: Revenue Per Employee at Flag

If we believe the overall cult revenue number of $180 million is reasonably close, and we believe that the cult has about 5,000 total staff, then we quickly compute an average revenue per employee of $36,000 per year.  If we assume that the cult has efficiently distributed staff across the organization (a laughable assumption, but bear with me), then if we take cult-wide revenue per employee of $36,000 and divide our estimated $50 million per year in Flag revenue by that figure, we get about 1,400 employees who should be involved in generating that much revenue.  That’s not terribly far off our estimate of about 1,600 total staff at Flag.

The point of this excercise is not that this calculation proves our estimates of either Flag revenue or of the number of staff at Flag.  It does, however, show that there aren’t obvious contradictions in our model.  If we had estimated 2,000 staff and $100 million in Flag revenue, that would imply revenue per employee of $50,000, substantially above the cult-wide estimate, and that’s particularly suspect in the case of Flag since many of those employees are in hotel and restaurant guest services, producing much lower-margin revenue than high per-hour revenue employees like auditors and reg’s.  In other words, our current model sanity checks reasonably well.

If They’re So Rich, How Come They Have to Do Their Own Construction?

The second interesting aspect of this is that, despite the claim of taking $75 million per year out of Flag in Clearwater, and a lesser amount on the West Coast, the cult is still so expense-starved that they have to turn some appreciable fraction of Sea Org into a construction brigade to save money on construction.  

There is no particular reason to disbelieve Jillian in this story, since there are plenty of accounts of waves of staff being demoted to the RPF when Big Blue needed renovations before, then mysteriously finding that they completed the RPF program soon after renovations were finished.

However, the fact that the cult is now using Sea Org staffers for construction duty suggests either low manpower utilization on useful tasks for the Sea Org, making it attractive to pull them to other tasks, or a cash crunch that would make deploying them on renovations important to avoid extra expenses.

Most businesses are somewhat rational in their decision-making process, deploying labor where it is best used either to generate revenue or to avoid costs (i.e., paying internal slave labor only a fraction of what outside experts would cost).  But it seems unwise to believe that the cult would be rational in deciding to deploy slave labor to avoid expenses, given that David Miscavige seems to make many decisions to enjoy arbitrary control over the lives of staff, rather than from economically valid principles.  The detail that Jillian was detailed along with another girl to remove Fiberglas insulation from ceiling crawl spaces in Las Vegas, rather than participating in a large-scale construction project at Big Blue suggests that the motivation for creating this construction brigade is unclear — a small project like might not necessarily have been undertaken for economic reasons, but just as a way for Miscavige to throw his organizational weight around.

When I started considering the apparent contradiction between Miscavige’s inflated claims of staff size and revenue at Flag versus the apparent need to save money on construction, I was hoping to be able to quantify this, but I’m not entirely certain about the financial aspects of this construction project. Anyone with any detailed perspective on recent construction efforts in Hollywood could help greatly in teasing out relevance, if any, of this detail.

The Russians Are Coming, The Russians Are Coming

The Russians Are Coming! The Russians Are Coming!
Poster for the 1966 comedy classic

Jillian’s assertion that 80% of Sea Org at Flag are Russians is quite interesting. There had been numerous reports that employees are increasingly non-native English speakers, and Eastern European origin for many of the staff was frequently mentioned, though Latin Americans are also a large part of the mix.  But the assertion that 80% of all Sea Org in Clearwater are Russians is a bit surprising.

Some discussions on WhyWeProtest suggest that these Russians are not all Russian nationals, but may be Moldovans and Russian speakers from Eastern European countries whose economies are in even worse shape than that of Mother Russia.  Interestingly, the economic situation in Russia is not all that good; I’ve read surveys that suggest that almost half of young Russian citizens want to emigrate to the West.  It’s not hard to believe that the situation is even worse in small satellite countries like Moldova or its breakaway Transdnistria region.  That certainly suggests that the cult has access to a pool of desperate recruits.

While it certainly cross-checks with already known facts that many Eastern Europeans are being sucked into the cult’s slave labor pool, it’s not clear that the number has reached 80%.  For that to have happened, there would have had to have been massive expansion of the size of the Sea Org.  There’s no evidence that there has been any such expansion, either in anecdotal checks in the activity levels at Pac Base or Flag, or in other stories from recent defectors.  Economically, given the low cost of Sea Org labor (I suggested the fully loaded cost of a Sea Org staffer is probably about $500 per month), the cult could potentially expand the size of the labor pool “just because” without wrecking finances. However, I am not sure the cult would do this just to feed Miscavige’s need to keep up appearances — adding 500 staff would cost at least $3 million per year, difficult to justify in a time where revenue is likely declining faster than the rate seen in previous years.

If massive expansion of the Sea Org was not the driver behind a sharp rise in the percentage of Russian-speakers in the mix, then the only other possible explanation is a significant wave of replacement of existing Sea Org members.  Again, there’s not any real evidence that this is taking place. 

I would thus treat the 80% number with some skepticism, as if it is a quick eyeball guess or a number relayed through a number of intermediaries before it got to Jillian.  It may be that 80% of new Sea Org recruits literally are Russian speakers, though the total number of Russian speakers in the Sea Org remains quite a bit lower.

If 80% of new Sea Org are Russian speakers, that would speak to the increasing difficulty of using the kids of existing members as Sea Org recruits. That, in turn, suggests that the potential to use the threat of disconnection from a kid in the Sea Org to keep doubting parents in line is losing its impact.  And remember, the parents of Sea Org members are presumably the more loyal of the older generation of Scientologists, and if a key retention tool to keep them enmeshed in the cult is losing its effectiveness, then further declines in membership in the short to medium term are inevitable.

Any further data points on the nationalities of arriving Sea Org members would be extremely helpful.

Emerging Trends to Watch

Eric Falkow joins staff... at age 79

Eric Falkow joins staff… at age 79!

Given the struggles the cult has in retaining staff, we’re always interested in looking at emerging trends in where they’re finding new employees.  This is why the influx of Russian speakers into Flag is important to monitor closely and to quantify more closely.

But in the last two weeks’ “Sunday Funnies” stories, Tony Ortega’s blog has featured older cult members going onto staff, the first time I can recall seeing something. Once is a curiosity, twice is a coincidence and three times might just be a trend worth considering.

Yesterday’s column featured the above ad, proudly announcing that Eric Falkow has joined the staff of the Pasadena Ideal Org.  According to one database I use, there is a 79-year old named Eric Falkow living in Pasadena; it’s reasonable to conclude that they’re the same person.

If we see any more older members joining staff, that could be evidence of yet another desperation strategy in trying to fill the rolls.  This and similar evidence of difficulty recruiting new staff leads to the scenario I’ll detail in a blog post later this week: can the cult continue to operate if the number of employees at local orgs shrinks beyond some minimal level?  

Analytical Techniques: Spotting Fallacies in Arguments

Impeccable logic + pointy ears wins arguments but does not necessarily help win the hearts of supermodels.

Impeccable logic + pointy ears wins arguments but does not necessarily help win the hearts of supermodels.

Summary:  We present a handy chart that an Alert Reader passed along with the most common logical fallacies in building arguments.  If you master and apply the material on this one page, you will amaze and impress (and probably also intimidate) your friends with your brilliance.  And by being able to spot errors in others’ analysis, you win the right to denounce their work with an air of haughty derision, and you spare yourself the embarrassment of potentially spouting inaccurate twaddle if you believe what they said.

Fair disclosure, though: your new-found command of logic may not make it easier for you to hit on supermodels (whichever flavor is most appealing to you) in trendy Manhattan clubs.  Don’t ask me how I know this.


An example of a Dadaist logical fallacy in action.

An example of a Dadaist logical fallacy in action.

The first place to check: One of the most important things to do in analysis is to spot fallacies in your own reasoning and in that of others.  It takes a lot of time to nitpick through all the data, making sure it is accurate and relevant to the argument.  But it is often not necessary, because logical flaws can undermine an argument and they’re usually much faster to spot.  

To become a more powerful writer, if you look for fallacies when you edit your work before publishing it, you can save yourself tons of embarrassment when one of your Alert Readers catches you at it.  And you will discover that you become more persuasive, as well.  Even if your readers don’t formally deconstruct your logic, people have enough understanding of logic that they’ll feel uncomfortable about your argument, even if they can’t specifically name the particular fallacy you used.

Honest mistakes:  Sometimes, people make honest mistakes in the way they construct an argument. That’s especially true in areas where people have a high degree of personal involvement, emotional or economic, to a particular outcome.

To make sure you don’t do this, it’s wise to learn the art of clearing your mind and pretending that you’ve never seen the document before that you just wrote.  If you can pretend that you’re about to edit someone else’s writing, it’s a lot easier to step out of the passion that guided you to write something, which may obscure logical flaws in your argument.  Passion is actually welcome in analysis, because it reflects the analyst’s level of conviction in his thesis, but it is only effective when it surfs on top of a logical and well-constructed argument.  Passion on top of logical idiocy gives political speech like we have today.

In the 1970s, Supertramp, which was a way cool band of the time, had a hit with "The Logical Song."

In the 1970s, Supertramp, which was a way cool band of the time, had a hit with “The Logical Song.”

Dishonest mistakes: Other times, people will intentionally use logical fallacies to “sell” a point of view that wouldn’t be supported by evidence. This is particularly common in politics. For example, many news organizations strive to report balance by reporting the political opposition’s view of a circumstance. When Democrats accuse Republicans of something, in many cases, Republicans will reply by talking about how Democrats do the same thing, or something they believe is worse. That’s a logical fallacy, and once you recognize it, it’s possible to keep the attention focused on the matter at hand.

Dilbert gets the relationship between logic, strategy and leadership right... as usual...

Dilbert gets the relationship between logic, strategy and leadership right… as usual…

How to Lose the Argument Before it Begins

There are several different categories of logical fallacies, relating to:

  • The structure of the argument (i.e., whether the reasoning to get from the evidence to the conclusion) is correct;
  • The evidence used in construction of the argument (too much anecdote, too little, etc.);
  • Intentional or willful distractions from a flawed argument (attacking the speaker, etc).

People intuitively know that an argument via an ad hominem attack is inherently a confession that the argument probably doesn’t have enough weight to stand on its own.  Few people are ever convinced by such argument that weren’t already true believers.  That’s why, though I may make legitimate mistakes in logic, I work very hard to avoid any of this deceitful, intentional reasoning error in anything I write.

And Now… The Poster!

Here’s the site: https://yourlogicalfallacyis.com

Here’s the poster: 

A great poster listing many of the common logical fallacies that people either intentionally or accidentally use in logical arguments.  From http://www.yourlogicalfallacyis.com. Click to show full-resolution image.

A great poster listing many of the common logical fallacies that people either intentionally or accidentally use in logical arguments. From http://www.yourlogicalfallacyis.com. Click to show full-resolution image.

Analytical Techniques: The Power of Anecdotes

Summary: We look at the role of anecdotes in researching the cult.  They can be powerful tools to either validate or challenge your existing thinking.  Anecdotes don’t prove trends or general conclusions, but they are a great tool for alerting you to possible trends, changes in direction, or conclusions you’ve missed.  This article talks about how we use anecdotes on Wall Street. But the best part is a case study, with one of our commenters reporting on a great chance encounter who interviewed a Scientologist at length in an airport bar, as well as my quick take on what to do next with an anecdote that challenges some of my beliefs about the cult.

Anecdotes are powerful tools: Today, I want to look at the power of anecdotal evidence in analyzing Scientology.   Stories from current and former members can be a powerful tool to check your assumptions and your thoughts about what is going on inside the cult. These are particularly important to help you make sure that reality has not changed without your noticing.  In other words, anecdotes that don’t fit into your current hypothesis of what is going on are one of the most powerful tools in improving your analytical work.

In order to make anecdotes work, one has to have a foundation of intellectual honesty. In other words, you have to be open to the possibility that some new piece of anecdotal data will unravel a theory, potentially even one that you are inordinately fond of.  You can’t rush to defend a theory without thinking dispassionately about what the new data point means. Pride in doing good analysis comes not in being right about a particular theory, but in being able to adapt your thinking and to continue to hone in on useful and actionable conclusions, even if they are heading in a different direction in your prior work.

While anecdotes are powerful, “the plural of anecdotes is not data.”  What I’m saying here is not at all conflict with what I have said in multiple comments on Tony’s blog and elsewhere about anecdotes as inherently insufficient to prove general conclusions.  As you may recall, I have said on numerous occasions that clear and convincing anecdotal evidence that Scientology auditing has produced big “wins” for some people in some circumstances is not sufficient to “prove” that auditing works in a general case across a broad population of people. As scientists say, “the plural of ‘anecdote’ is not ‘data.'” That’s because anecdotes, no matter how credible the teller, aren’t structured rigorously the way that statistically valid data points in a clinical drug trial would be.  So you can’t get from “a big bag of positive auditing success stories” to the assertion that “auditing works and is an effective form of therapy.”  

In other words, anecdotes are great ways to get you to continually challenge your existing views and to guide your work by digging deeper into inconsistencies in your scenario of what is happening and your predictions about what will happen.  For that, one or two anecdotes can be sufficient to open up a whole new area of research.  However, those same anecdotes are not proof of your new theory or model.

Incidentally, I am working on a longer piece that looks at the apparent contradiction of how anecdotes can be valid individually, but any number of them cannot be combined together to establish a true statement.  It should be out in a week or two.

How anecdotes make you rich and famous on Wall Street:  In the late 1990s, Oxford Health was an HMO growing explosively, and the stock was on a rocket ride.  But one analyst, who checked in with doctors who were Oxford providers, started to hear that they were having trouble getting paid, though she knew that Oxford had always been very timely in physician payments to date.  She talked to more doctors, did some more research, and eventually made a gutsy call: Oxford would miss their profit forecast for the quarter for the first time ever, and by an immense margin. Her research helped get her clients out of the stock while it was still high and avoid catastrophic losses when the company reported several weeks later that they were hemorrhaging money and the stock collapsed.  An article from the New York Times talks about the Oxford case (I can’t remember the name of the brave analyst who went against the grain and was roundly criticized until she was proven magnificently right).  And this article from the Wall Street Journal at about the same time gives more depth on the thought process of using anecdotes in a very powerful way.

Case study:  Let’s consider a case of a really interesting anecdote which was sent in by “B. B. Broeker,” a longtime commenter on Tony’s blog.   He ran into a Scientologist at the airport in Tampa and had a long chat with a longtime supporter of the cult, which he relayed to me.  He said:

I was in Tampa for business not long ago. When my business meetings went more smoothly than I’d predicted, I saw my chance. I drove across the bay to Clearwater, parked near the Super Power building, and took a leisurely walk around the Scientology complex. It was a pretty unremarkable visit, but I was glad to have seen up close the buildings that have occupied so much of my mental real estate since becoming a Scientology watcher.

On my way home, I stopped at the airport bar, and sat next to a chatty woman in late middle age. She was, based on her interaction with the bartender, on what I figure was her fourth or fifth glass of chardonnay, and was engaging the guy on the other side of her in a trite conversation about the deleterious effect electronic gadgets are having on communication.  Needless to say, I stayed buried in my phone.

While I avoided a conversation for a while, I eventually gave in after she directly asked me how my (crappy) food was.   As it happens, I was reading Mike Rinder ‘s blog when I finally surrendered.  It turned out she lived in the greater Clearwater area, and I mentioned that I’d just been there. She named a couple restaurants and asked if I’d gone to them, and I said no, I’d just visited on a pilgrimage of sorts to the Scientology complex.

Her jaw sort of dropped, and I figured, “oh, shit, she thinks I’m a clam, and doesn’t know what to say.” So I hurried to add, “yeah, I find them fascinating.” She fumbled a bit, and eventually said, “you have no idea what’s about to happen there.   I’m a Scientologist.”

Now it was my turn to be taken aback, but I quickly recovered.  “Yeah! Super Power is finally opening! The IAS gala! Golden Age of Tech Phase II is debuting! And … you’re leaving town?”

She seemed suspicious, but answered. “Yeah, I’m headed out of town for a while. I’ve got lots of friends [at my destination], and I need to get away for a bit.”

(beat)

“How do you know all that, about all the events?”

“Oh, I read a lot. Like I said,  I find your religion fascinating.”

Well, after telling her what I do for a living (I was soooo tempted to say I was a psych, but I made a conscious decision to not antagonize her, both because I didn’t want to be mean and to see if I could get this tipsy woman to open up), she seemed to decide that I was good people, and she told me her life story.

She grew up in one of the richer suburbs of a large city, but her family wasn’t really wealthy, and she didn’t really fit in with the other kids.  Consequently, she had a hard time of it in school. “I didn’t need Scientology to teach me how to stop being effect and start becoming cause.  I had to learn that in high school.”

She got into Scientology in her 20’s.  Her boyfriend introduced her to the church.  They got married, and her new husband started a company which he ran on LRH “admin tech.”  It succeeded, and was later sold, and they moved to Clearwater.

It was at this point that she confessed that he wanted a divorce, and that she felt like she needed some time apart to figure things out.  That’s why she was headed out of town.  He wanted to stay for the events, and she decided to let him have them, while she got her head straight.

“I’m really sorry to be missing what’s happening – especially the developments in the tech and processing – but I can watch them all on DVD when I’m [at my destination].”  I guess she was planning to be gone quite a while.

We talked about the tech, and how much it helped her and her husband relate better (I courteously ignored their impending divorce), and how study tech is probably the greatest advance in human development in the past thousand years.  She even talked about the amazing efficacy of Narconon  – she had referred family members to the center and tried it for her own drinking problem.  She felt the tech and the counselors had saved their lives.  (I chose not to comment on how she was throwing back the vino – probably on glass five or six – at that very moment.) She volunteered that her husband was on a fairly high Bridge level, and had been for a number of years, but I didn’t know if it was a faux pas to ask about her own case, so I didn’t.

Anyway, I continued to demonstrate I was knowledgeable about the subject, so I wasn’t that surprised when she said, “you know so much about Scientology. Have you ever taken any courses?”

“No, I haven’t.”

“Why not?”

“Well, I’ve read a lot of LRH, and … well, I guess it’s just not for everyone.”

She sort of accepted that, but after a while eventually returned to the topic – not in a proselytizing way, but as if she were genuinely curious why someone who had familiarized himself with the Founder’s work wouldn’t want to rehabilitate his spirit.

“Is it it the press?  You know you can’t trust the papers.”

“Oh, I know.  But I agree with LRH – ‘look, don’t listen.'”  (She smiled wide at that.) “I just don’t feel like I need Scientology. “

Again, she seemed to accept my position, but then she asked me a question I never would have expected:

“Everyone thinks we’re crazy, or we’re weird. I mean, people seem to hate us. You don’t – don’t get me wrong.  But why do you think people hate Scientology?”

It was touching, and not a little bit sad. She really wanted to know, and really had absolutely no idea, why the vast majority of people outside her little bubble believe that something at the core of her life is ridiculous and/or contemptible.  In keeping with my approach of not antagonizing her, and because I thought it would lead to a more illuminating discussion, I played it soft:

“Well, there’s the money aspect–“

which prompted her to talk about how much training the auditors all had, especially with the GAT II release and with Super Power, and about how that costs lots of money, and there’s all sorts of self-study courses besides.

“Right, but I wasn’t talking about donations for coursework or auditing.  I mean, the fundraising.  The Ideal Orgs.  The IAS.  You’ve been in for eons – do you get the sense that they’re regging you harder?”

“Well, maybe. But they really don’t pressure you to give what you can’t afford.  I’ve never felt pushed to give more than makes sense. Sure, really wealthy people – and there are a lot of quietly wealthy people in the Church – give a lot, but it’s nothing to them.  Normal people aren’t forced to give that much.  It’s just not expected”

“Are you guys IAS patrons, or anything like that?  Did you get pushed to prepay for Super Power?”

“No, we’ve got two bridges to pay for, and college for the kids.  We give what we feel we can, but our bridges come first.  And no one makes us feel bad about that.”

I don’t know about you, but I found that fascinating.  Sure, it could be a PR line, but it was delivered pretty genuinely, by a woman who had heretofore demonstrated no ability to effectively shade the absurd disconnect between her idealized vision of the tech and the reality of her experience in the Church. (See: her impending divorce, her Narconon “success” story.)  Now, whether she actually isn’t being coerced into donating, or whether she no longer can discern coercion – whether she actually isn’t giving a lot to the IAS, or whether she no longer has a sense of what “a lot of money” actually represents – I don’t know.  But I believe that *she* believes that there truly isn’t a regging problem.  And that’s interesting in and of itself.

Anyway, we chatted for a little while longer, but I soon had to head to the gate. As we parted, I caught her name off her boarding pass.  I checked her on Kristi Wachter’s completions list, and she had indeed been in the Church for quite a long time.  And I suspect she’ll never leave.

Thanks to BBB for a well-written narrative, and for doing a great job helping the lady he was talking to to open up.  Great job on sucking up the snark and wit to ask bland questions to help her feel comfortable.

How to analyze this data point:  There are  a couple areas where the lady’s statements fall outside my beliefs about how the cult operates.  Here are what I noticed and how I’d react to them (not to refute her statements, but to dig deeper to see what’s really going on):

  1. Regging is at tolerable levels:  The lady says that she doesn’t feel overly hounded for money, even though she is reasonably well off in semi-retirement, which I would assume makes her a prime target for enthusiastic FSM’s.  Given the horrific stories that have emerged from so many quarters, I’m surprised to see someone who is relatively sanguine about the amount of fund-raising in the cult.  It’s not likely that all those stories of obscene fund-raising techniques are wrong, but this lady apparently spoke truthfully (“in vino, veritas”?) about how she doesn’t feel overly pressured to donate all the time.  There are several possible explanations, and we would need further follow up to determine which might be applicable:   a) her husband might be the target of all the regging, since he controls the money in the family; b) they’ve reached the status of a “sideliner,” having made clear to the cult that they’re not giving more money ever; c) the cult is toothless to follow up on e-mails sent out in order to get people to attend events; d) the cult is more sophisticated in fundraising approaches, spending less time on members who are assessed as less likely to give, or e) something else entirely.  A detailed follow-up interview, if it were possible, with suitably gentle and wide-ranging questions might be able to give some perspective.
  2. Focus on the “Bridge” instead of events and donation: the picture in publicly available testimony is that the cult is making it difficult for people to move up the Bridge because it’s forcing them to redo long-ago levels and courses.  The fact that so many recent escapees say that having to redo “Objectives” caused them to blow may be a function of a self-selected audience; we’re not interviewing people still in the cult (which is why this conversation is so interesting).  I am intrigued that this person’s story challenges what many of us take on faith about lack of progress on the “Bridge.”  I would want to ask a whole bunch of follow-up questions including understanding how much Bridge progress they’re making, and whether they have had the setbacks (kicked back to “Objectives”) that others complain about. In other words, are they just engaging in a little cognitive dissonance, like touting the benefits of Narconon while belting back the drinks?  Or is there some sophistication in how the cult is targeting its members to maximize the total revenue per customer (like a casino who knows which customers prefer blackjack to poker, so they don’t shoehorn a craps player into a roulette game that they don’t really want to play).  Or, again, is something else in play?
  3. Narconon:  the fact that this lady was quickly getting bombed while talking about drug and alcohol “tech” is amusing. But beyond this, it’s reasonable to guess that a possible reason she’d be doing something that most addiction and rehab experts would say belies any actual rehabilitation, is that the cult’s definition of “recovered alcoholic” differs from the one used in the rest of the world by a fair margin.  In other words, the cult may rely on definitions to get people to think Narconon works.  She may think that because she has completed Narconon that that is what determines whether she’s an alcoholic or not.  On Tony’s blog the other day, a commenter quoted a story of one cult member saying of a nearby OT VIII who smoked madly, “he could quit at any time, he just chooses not to.”  It would take a follow-up interview to see if the lady believes Narconon works because it teaches you that you have the power to stop drinking any time you want, but that doesn’t mean you necessarily have to stop today.  If that is indeed the definition culties use for “success” at Narconon, it’s no wonder it’s easy for them to repeat claims of an 85% success rate for the program with a straight face.

The more data points one collects on a regular basis, the better prepared you are to detect changes in the environment that would allow you to update your scenario. The faster that you detect and respond to change, the more effective you’ll be… in capitalism, if you figure out that a company’s business is deteriorating, you can sell the stock before others think there might be a shortfall, and can often avoid huge losses.

Analysis Versus News Versus Opinion Versus Lulz

As I said in my first post on this blog, I want to try to add value to the world of Scientology activism by building a site for deep analysis, initially my own and, over time, building a community of like-minded folks to work together.

Why?  Because analysis is the prelude to action.  If you can do the following:

  • Understand the problem better
  • Identify a wider range of possible solutions
  • Predict accurately the potential results of each different solution scenario, especially by minimizing the chances that you’ll miss something leading to unintended consequences,
  • Apply enough analytical rigor to choose the best solution, and
  • Do that as quickly as possible

…you can significantly improve the outcome of whatever you’re doing, whether that’s winning a war, beating your competition in business, picking stocks, or doing any one of a hundred different things in a dangerous, competitive world.

What is Analysis?

Most people don’t understand what analysis is and where it fits in the different ways to think about Scientology. In this post, I’ll try to position analysis against the other types of writing that people have done about the cult. I will also try to show how deep analysis can help us work together more effectively as we try to oppose the evils of Scientology. 

Analysis is the prelude for action.  It’s not a pointy-headed intellectual exercise.  If you can gain some unique insight into the problem you’re solving, you can identify a broader range of possible solutions and arrive where you’re headed faster and more efficiently than your competitors.  Long-term competitive advantage is a function of doing this better than other people a little bit at a time for years at a time.

Fundamentally, analysis is based on facts, which are then refined by experience, logic and judgment, into a testable series of predictions about the future. These predictions, unlike predictions from theories of physics or other sciences, are not necessarily exact. But they should be testable, if for no other reason than to make sure that the analyst has some skin in the game to be accountable for the quality of their work.  The end product is an opinion but it is a highly credible opinion.  And it is an opinion that still has value even if the scenario predicted doesn’t come to pass. 

A great example of the positive impact of good analysis is in the world of building software.  In the classical “waterfall model” of software project management, projects move through five steps: analysis, design, implementation, testing and post-release maintenance.  A well-known truism says that it costs 10x as much to fix a problem in a given stage of the process than it does in the prior stage.  It’s also true to say that an error in analyzing the problem and potential solutions is 100x more expensive to fix if it’s discovered during implementation.  So in that domain, the economics of good analysis are compelling.

The Elements of Good Analysis

Among the key elements of good analysis are:

Testable conclusions:  The end product must include clear, unambiguous predictions of some future event, in a way that the conclusion will be verifiable. You have to be able to determine whether you were right or wrong. 

A statement that “Microsoft will have a good quarter” isn’t terribly useful, nor is it testable.  On the other hand, consider this statement: 

“We now believe Microsoft will report earnings per share of $0.03 to $0.05 above the current consensus earnings estimate. We expect upside to be driven by Internet search revenue coming in approximately $100 million above our former estimate, but more importantly, by lowered capital expenditures in that unit, coming from tighter expense controls.  If we are correct, we expect the price/earnings multiple on the stock to expand slightly as investors are relieved that the Company is becoming more consistent in applying expense controls, and we would expect the stock to rise from the current $37.66 to around $39.50 in the short term after the earnings release.”

That’s a highly testable scenario.  It gives five specific predictions, and each one of them can be clearly examined to determine whether they’ve come to pass.  There are hints of what might go wrong if the prediction fails — the stock won’t go up in price even by the 5% we are expecting here. 

Arguments and data that add value even if the conclusion is wrong:  Analysis is not like betting. You have to accept that even your best work won’t get the conclusion right all the time.  And you also have to accept that you might get the conclusion right but your scenario showing how you got to the conclusion will be off, or vice versa.  It’s like a complicated problem in algebra: you get marked off a little if you get the answer wrong, but you lose all the points if you get the answer wrong and you don’t show your work.

I have been recognized by peers on many occasions for doing great work even if I’ve gotten the conclusion wrong, because my thought process provided a new way to look at the operations of a given company, or because I was able to discover some data about the company’s operations that no one else had.

One of the key things about formulating a good argument is the notion of intellectual honesty. It usually ends badly if you formulate an argument designed to reach a particular conclusion, but you intentionally omit facts that undermine your case.  In my business, it’s a certainty that anyone who tries to hide known facts to take a shortcut to making their case will get caught by some sharp-eyed client who will then dismember you.  It’s like a district attorney prosecuting a case: he can’t hide evidence from the defense that would contribute to finding a defendant innocent.  In other words, he has to prove his case while disclosing as much information as possible to help the opposition prove theirs.

It’s similar here.  Intellectual honesty demands that we acknowledge what Scientology has done right as well as attacking what they do wrong.  After all, the cult survived the imprisonment of almost a dozen members of senior management following Operation Snow White in 1977. It survived several epic legal setbacks in the 1980s and 1990s, and it survived the Mission Holder Purge among other changes when David Miscavige took over.  They had to have been doing something right to stay alive then, when other organizations would have been mortally wounded. Thus, the “cult collapse scenario” (first version yet to be written as of publication date) would have to acknowledge and account for why those events were not fatal in order to be credible in establishing that the current scenario may actually result in the cult shutting its doors forever.

The right amount of rigor for the available time frame: Analysis is sort of like science, with one difference: with science, predictions have to be right all of the time.  Kepler’s laws of orbital mechanics work 100% of the time.  That’s why those are laws and the process of deriving them is science.

Analysis tries to apply science-like rigor to much squishier problems than physicists might try to solve.  But every problem is different, and part of the job of a good analyst is to figure out how much work to put into a solution.  A highly precise answer a day late is useless on Wall Street, though that may not be much of a problem in other types of analysis, like legal research or drug trials, where safety is paramount and the length of time to complete a trial is secondary.

So part of the process is figuring out how much rigor you need.  Sometimes, back-of-the-envelope estimates are all you have time for, but even those have a certain amount of rigor behind them; I’ll cover back-of-the-envelope exercises (which are a lot of fun) in one of the first articles I do on analytical technique.  Other times, you need detailed analysis that covers a substantial portion of the sample space, and analysis approaches science in needed rigor.

A commitment to improving the process: Because we’re dealing with a human process, it’s important to think about how to improve the process. After every failure, and after every success, it’s important to debrief and spend at least a little time to figure out what worked and didn’t work.  I try to do this with analysts on my team, particularly after a notable success or failure.  This is one of the key lessons of the aerospace industry: after every crash, the government investigators work to figure out the cause of the crash, and then work to ensure that no future crashes occur because of that particular problem.  I’ve written tons of mea culpa research reports to clients after I’ve blown a call in a particularly egregious way. Nobody expects me to be perfect; they just expect me to be diligent, and one of the ways you show your diligence is by accurate self-analysis when you go wrong.

An ability to admit you’re wrong: A big part of being a good analyst is to get your ego out of the way. If you understand that the goal is to do good work, creating useful conclusions as well as well-supported arguments to get there, then you’re doing a good job. Staying humble in the face of the difficulty of the task helps.  Admitting you’re wrong and taking clearly visible steps to get it right next time is key.  And having compassion for others who get it wrong after doing a good job is important.  That’s why avoiding the easy temptation for ad hominem attacks is really important if we’re going to do serious work here.  I have successfully managed people I dislike intensely on a personal basis, but if they’re doing the work thoroughly and honestly, and if they admit they’re wrong, I will treat them professionally and respectfully and I will pay them well, even if I am invariably ready to smack them when they come into my office to complain about something.  

There’s an unspoken agreement that’s necessary for survival in stock picking: criticize my work, don’t criticize me. As long as I’m diligent and useful, you do not have the right to engage in an ad hominem attack.  Conversely, in my world, leading with an ad hominem attack is seen as a virtual guarantee that the attacker has no game and is engaging in a very poor attempt to cover it up.

Analysis versus News

News and analysis are highly complementary but they are not the same.  We in Global Capitalism HQ depend on accurate news reporting about companies and about external events (Fed actions, etc) as a cornerstone of investment decisions.  We spend a certain amount of time thinking about whether a given news story is credible, and we have opinions about certain reporters in our respective areas of expertise.  I loved certain Wall Street Journal reporters and trusted them implicitly, but others I wouldn’t trust as far as I could throw them, because they had areas where they were completely unable to understand the real issues they were covering.

News is about answering the classic questions about an event: who, what, where, when and why? It’s about delivering the most relevant facts about the past. A news reporter’s job is to focus on the past, and not to get distracted by the future. Typically, of course, a good news reporter will interview experts who do have an opinion about the future and will quote them extensively in looking at the significance of the event. But it is not the news reporter’s job to express his own opinion about the future, even though a good reporter is close enough to the story that he will undoubtedly have an opinion.

Tony Ortega is a good journalist because he goes to appropriate lengths to keep his personal opinions out of the story. Sure, Tony has opinions about the issues he covers; that’s how he is able to figure out when a particular source is lying or telling the truth.  In journalism, the writer is not allowed to say what he thinks, though he may slant the story by focusing more on sources on one side of the issue versus another. The degree to which reporters do this honestly factors into their long-term credibility, of course. 

In looking at whether a news reporter is wrong, it is fairly straightforward to determine: did the reporter ignore or omit key available facts in reporting the story that would have been available to a competent reporter?

Getting a news story wrong is a big problem, but in analysis, as long as you put in the work and do your best work, it’s disappointing but not fatal.  In that respect, analysis is like baseball.  In the 2013 season, Detroit’s Miguel Cabrera led the league with a .341 batting average.  That meant he failed to get on base almost two out of three times he stood at the plate, but he was better than anybody else last year.  But football is more like news reporting: getting a story wrong is bad for the career in much the same way as missing a field goal from the five yard line would be in the NFL.  You’re just not supposed to miss those if you’re in the pro’s, even if it’s 15 below and snowing like mad in Green Bay.

Analysis Versus Opinion

All analysis is opinion, but relatively few opinions count as analysis.

Everyone has opinions.  Most people tend to think their opinions are inherently damned smart ones, otherwise they would have different ones. In other words, their egos are tied up with opinions.  That’s fine when you’re thinking about your favorite basketball team or which one of the supermodels is the most beautiful.  Those are hard arguments to win with facts.  But it’s really dangerous to treat opinions as true when you’re trying to make your opinions bring about something big in the world.

You’ll notice that I work very hard on Tony’s site to qualify statements I make as verifiable fact, as analytical opinions with a high confidence level (often indicated by reference to a document, or with terms indicating my confidence level), as a hypothesis, as an educated guess, as a supposition, or as merely a speculation (almost a fantasy).

For example, in a comment on Tony’s blog today, I observed that if Tommy Davis really was in NYC, as part of his job traveling as an assistant to the chairman of his real estate investment fund, it seems likely that he’d be spending a lot of his time in LA, but if his wife is still living in Texas, is there a possibility that they’re divorced?  I specifically said that this was a potential inconsistency in the facts that bears further checking, and that it was not intended to start a rumor that they’re divorced, and that it certainly wasn’t fact.  I clearly stated my level of confidence in what I wrote — which was zero, since it was really a question.  That’s really important in analytical work.  The CIA and other intelligence agencies have a very specific standard for the verbiage they use to express their confidence levels in various things that they write.  We on Wall Street are a bit less formal, but it is still important.

Analysis Versus Lulz

All of what I’ve written above has been deadly serious.  But that’s not to imply that analysis is inherently serious, and indeed, boring.  If you can be funny in a research report, it sure helps to make your conclusions jump out at the reader, if the humor is built on solid logic.

And sometimes, the lulz that the Anons post can actually point the way to finding out and identifying deeper truths about Scientology.  The finely honed sense of the absurd that keeps many Scientology protesters engaged can actually help give insight into the cult, particularly how its members behave.

Though a lot of what I’ve written so far on this blog has been serious, I’m not like Sam the Eagle from the Muppets, utterly humorless and perennially serious.  In fact, over my career, I’ve been admonished to dial down the humor far more than most people that do my job.  Here, I’d like to encourage wit and humor to be part of what we end up doing, though not if it comes at the cost of good thinking and good data.

Outside References

There’s not much written about an introduction to Wall Street analysis, since the field changes fairly rapidly, and since most people just learn on the job.  But it’s fun to read about military analysis and try to understand the general thought process, even if the mechanics of producing military and political intelligence is way too cumbersome for what we’ll try to do.

  • The Wikipedia page on Intelligence Analysis gives a pretty good overview of the research process in a discipline that is more rigorous and slower-moving than what we’re trying to do.  It’s a good place to start to indulge your inner James Bond as you think about how to add rigor to what we do.  I’d suggest reading this and looking at some of the referenced articles, such as the NATO Open Source Analysis primer below.
  • The NATO Open Source Intelligence primer gives some understanding of the process of “open source” intelligence (not the same as open source software), which uses publicly available information such as newspapers and broadcast media to figure out what a potential adversary’s intent is.  It’s all about looking for patterns in disparate sources and having a good process.  Unfortunately, this document is written in Eurocrat-speak, which is even worse than most military bureaucracy, and it’s a bit dated, but still useful.
  • A bibliography page at the USAF “Air University” training center has a ton of reading on intelligence and related topics.  A lot of the books in the sections titled “Intelligence Basics” and “Intelligence Analysis” are classics in the field.  Knock yourself out on some of these.  Of these, I’d particularly recommend:
  • The Psychology of Intelligence Analysis, by Richards J. Heuer, Jr., published in 1999 by the CIA and released in the public domain.