The Black Swan: a book review of sorts
Tuesday, July 31, 2007
Over the last 2 days I read
The Black Swan: The Impact of the Highly Improbable, by
Nassim Nicholas Taleb (partly research, party self interest).
Taleb's premise for the book is the effect outliers have in everything around us: these outliers are what he calls Black Swans. For example, a system that appears to be fully predictable all of the sudden displays an unexpected event dramatically changing the whole system.
In the book there are two worlds: Mediocristan and Extremistan. According to Taleb, we live in Extremistan and the Black Swans are what make our world go round. There are many examples of extreme events, for example, Bill Gates' net worth: he is worth around $60 billion, the rest of us are not, and are more or less aligned around a more realistic average. This is one of the books main points, as building averages with this type of outliers is what makes Taleb's argument valid. In a world with Bill Gates and John Does around, the average measure of wealth in our society is meaningless. Imagine the average of a population of 2, with Bill on one end and John in the other: ($60,000,000,000 + $10,000)/2 looks good on paper--the average would suggest that the whole population is doing very well.
Hence, our world is Extremistan, where not everything can be averaged or meaningfully represented by well defined statistical distributions such as the Normal distribution.
To explain how we are blindsided by Black Swans, Taleb presents the example of a turkey that is fatten up for 1000 days, and then on day 1001 (it's thanksgiving) there is no more food, as the turkey is cruelly decapitated and scrumptiously cooked.
To us, the feeders, this is not a Black Swan, as we know what's coming to the poor turkey. To the big ugly bird, though, this is a Black Swan: it was fed for 1000 straight days and there was no indication the 1001th day would be any different.
It is this lack of predictability of events that drives Black Swans. Taleb argues that our inductive methods of proof fail to predict Black Swans because we are only looking at well behaved data: our methods can't look or predict outliers. More specifically, if we were the turkey of the story (and we are in so many ways, e.g., we can't predict how many people will die in a war), we rely on historical data to live our simple eating, fat lives.
I don't think anyone will disagree with Taleb. He admits, though, he is not the first one to point this out. His beef, however, is with our reliance of mathematical theories and models to try to put order in a random world. For example, he really doesn't like CAPM, or the French, or well paid CEOs, or Economists, or guys who wear suits and $250 ties, or tenured faculty, or Nobel price winners. In fact, his insistence on being sarcastic and jabbing the French every fifth page is distracting. And he doesn't tell us why he doesn't like the French. On a personal note, I don't know "The French," but I don't dislike them.
Overall, I found his writing interesting, otherwise I wouldn't have read the book in two days. Nevertheless, I could have read Chapter 15, 16, and 17, and still have gotten the point; though, he does tell the "technical" reader to skip these very chapters. To him, those who have been trained in the sleazy arts of mathematics or finance are already tainted with bias.
He calls everything we do a Gaussian fraud, because of our over-reliance on the Normal distribution. But, to be fair, not everything has a normal distribution and he knows it but he doesn't clarify it throughout. I'm also puzzled why he doesn't mention the Central Limit Theorem. Perhaps there is no need to make his point, but I didn't like the omission.
Furthermore, Taleb calls himself an empiricist: he says he tries things and then tries to apply models. The rest of us do the opposite: we are top-down users, as we rely on mathematical models to play with nature. He has been a derivatives trader, and perhaps made a fortune while doing it (he's retired), so maybe he knows what he is talking about. Personally, though, I haven't gone through a finance course where all these mathematical methods he shuns don't come with a big
if attached to them. There is always the disclaimer that "sure, these methods work on paper, but in the real world they don't so well." No professor worth his tenure would claim that the mathematics work in every scenario.
So why do we study all these useless concepts? That's also a question he asks, and perhaps his source of disgruntle (if you read the book, you will see what I mean): I think he wants his theories to become mainstream. The problem is that, as he was told, we can't just change everything in one day, and, most important, "...you cannot throw the baby out with the bath water."
He does present an alternative, based on the work he is doing with Benoit Malderbrot (right, the
fractals guy). He vaguely explains how these outliers can be incorporated into a new model. The problem I see, however, is that Black Swans are not really predictable: they are just outliers, which are highly improbable events. For example, I can't turn invisible but it doesn't mean that it is impossible--just highly improbable. And this is the point of the statistical models we do study: should you be basing your future on something that is not likely to happen in the history of the whole universe (that's way more than 6000 years, as some believe). Probably not. Nevertheless, being able to predict the next famine or the location of the birth of the next suicidal dictator is probably a good thing, though unlikely to be predictable.
He does write early on in the book that if a Black Swan is not observed, it probably doesn't mean that it wasn't going to happen; but it could mean that the outlier was prevented by some random act of heroism. In the world we live in, our heroes are not the steroid pumping athletes we see on TV, but the ones who kill the butterflies before flapping their wings on the other side of the world, i.e., the ones who prevent the storms without knowing.
I do believe there should be a more technical book, a book that will not make the New York Times bestsellers list, but a book for the rest of us: the ones who don't skip chapters 15, 16, and 17. (I've looked for the mathematics, but I've only found
introductory articles.)
As far a recommending the book goes, I'm a bit conflicted. It is an easy read and is the popular book of the times (similar to Wikinomics or The Tipping Point), but it left me wanting more (perhaps it's the main goal).
Nonetheless, I think you should contribute to his retirement fund: buy the book, read it, and impress your peers with your knowledge of fat turkeys and their limited intellect that doesn't allow them to predict that they will be eaten in important holidays (we would know, right?). Also, by buying this book the chances of a next book increase: I'm expecting his next tome to have an explanation of why he dislikes
The French, and Nobel laureates, and fund managers, and...(Well, he kind of alludes to the fact in the book, but maybe I skipped over that part.)
Who Knew Digg Had Avertising?
Wednesday, July 25, 2007
Because I'm so used to Firefox filtering all that bandwidth
wasters, I had forgotten digg had ads on its pages.
It will be interesting to see what the world will look like with Microsoft and Google
fighting for the same turf.
Note: to block ads, I use
Adblock Plus. Obviously, you will also need
Firefox. Give Firefox a try...
Canada Post delivers the goods
I wrote
yesterday that my package from Amazon was delivered to the wrong address.
Fortunately, I received my package today.
The problem: human error. I live in unit 906, but my package was delivered to unit 908. And when I say human error, I don't mean the mail carrier's error; I mean someone typed in the wrong unit number.
You would think that with every label being printed in a non-serif font mistakes like this would not occur. Not so. Along the way someone had to actually read off the 906 of my address and transcribe it to print off a different label. The new label reads 908.
I'm not sure why there is a need for two different labels, if the first one already has
all my information together with the actual tracking number. In theory, to save money and the environment, one label should be enough.
Label 1:
Label 2:
Label 1 and 2:
To be fair, that 6 looks like an 8. But my questions still remains: why are there two labels with the same information on the same package? Sure, it costs pennies, but multiply those pennies by hundreds of thousands of packages a year and they add to a considerable amount.
I don't know anything about Canada Post's processes, so there must be a very good reason for that extra label and extra labour to exist. But from the layman's perspective, it doesn't make sense--unless you want to loose packages by misprinting one of them :)
All and all, I'm satisfied with my ecommerce experience.
Where is my stuff, Canada Post?
Tuesday, July 24, 2007
On the weekend I ordered 4 books from Amazon.ca. I have ordered from them before, and the whole transaction is painless: I pay, I receive my order. In most occasions (every time so far), I use their free shipping option, as Canada Post is very efficient now a days and any ordered items are in my mailbox one or two days later--this is very good, as I pay nothing and get next day service.
Today, however, there is a problem: Canada Post's tracking system indicates that my books are already delivered, but I didn't receive anything, which leads me to believe that my package was delivered to the wrong address, or their tracking system is broken.
I don't know if a mistake has been made or not, as Canada Post's customer service is already closed (8:30 AM to 6:00 PM). I can, as their voice message indicates, check their web site and automated phone tracking service for the status of my items: the problem is that my items are marked as delivered and I can't get any more information.
I have to wait until tomorrow to track my books.
If you are curious, one of the books is
The Black Swan: The Impact of the Highly Improbable. I just completed a graduate statistics course on forecasting, and I'm writing a research paper on Technology Forecasting and Technology Foresight so I have developed a taste for the predictability of events. Of course, predicting the future on past data is very inaccurate if not impossible, unless there are clear trends or predictable seasonality patters. Whether any forecasting methodology works or not is up for debate, but in the mean time, forecasting is a booming industry: someone has to predict interest rates, the weather, and equity prices.
I mention this particular book, out of the four, because of the irony: how likely is it for these books to get lost in transit?
The probability is high considering all the possible fail points, but it's likely improbable because of all the checks and bounds of where and how this package is traveling. In other words, if I can't track the location of these books, it will be an outlying point on Canada Post's delivery record (if it isn't, it should be).
I'm sure the books will appear somewhere, or at least I hope they do. Otherwise, someone will be having a ball with the writings of
Taleb and
Dawkins.
Blade == 0.45 x The Matrix
Monday, July 23, 2007
I had never seen the movie Blade, until yesterday. It's a good movie, and there are some similarities to The Matrix that I couldn't ignore. I can see the inspiration, but
minus add the philosophical ramification of living
inside a computer and remove the vampire storyline.
Nick Bostrom, an Oxford philosopher, wrote in 2001 an interesting paper that introduces the idea of why we are most certainly a living and breathing computer simulation. From the paper's abstract:
This paper argues that at least one of the following propositions is true: (1) the human species is very likely to go extinct before reaching a "posthuman" stage; (2) any posthuman civilization is extremely unlikely to run a significant number of simulations of their evolutionary history (or variations thereof); (3) we are almost certainly living in a computer simulation. It follows that the belief that there is a significant chance that we will one day become posthumans who run ancestor-simulations is false, unless we are currently living in a simulation. A number of other consequences of this result are also discussed.
The arguments are not new, but modernized into our transistorial world. For example, have you heard of
Plato's cave?
Soccer in Canada
Saturday, July 21, 2007
A good assessment of Canada's performance in this U-20 FIFA World Cup:
FIFA U-20 World Cup shows problems facing Canada as soccer nation
Backwardation
The English come up with all sorts of funny terms, and of course we have to learn them, for we have adopted most of their financial machinations (e.g., stock market).
Contango and backwardation theory deal with spot and futures prices of commodities. For example, contango refers to the spot price of a commodity (say oil) being less than the futures's price; backwardation is the opposite: the spot price is greater than the futures's price.
The reasons for the spot price being higher or lower than the futures vary: different aspects come into play such as carrying cost and convenience yield (fancy words to talk about inventory cost and a bit of profit premium for you having the actual commodity).
These are not your everyday concepts, but are useful nonetheless--arguably, only useful to finance geeks or investment managers.
For a more or less in-depth definition, visit
Econbrowser .
Canadian Police vs. U-20 Chilean Team Video
Friday, July 20, 2007
You have to love the internet...
For those who don't speak Spanish, the announcer is saying that the Chilean players were walking towards a group Chile followers to thank them for their support; at this point, the players and the team staff were not allowed to get close to the group and that's when they were detained and handcuffed. (This is from a Chilean TV station.)
A pattern or just coincidence?
I shouldn't say it's a pattern, but the title is catchy, isn't it?
I'm talking about the two national Chilean teams getting in trouble in Copa America and the FIFA U-20 World Cup:
This is not good news for the Chilean soccer federation.
Tasered and handcuffed, eh?
Why leave Canada without getting a little love from the Toronto police? That's probably what U-20 soccer Chilean players thought.
I wasn't there, but from what some
newspapers reported this morning, 9 Chilean players were handcuffed, some tasered, some pepper sprayed, and some roughed up for disorderly conduct.
Chilean officials, of course, are pissed about the whole thing. And who wouldn't be? But breaking the windows of the bus was probably not a good idea.
I'm sure that in the heat of the whole confrontation mistakes were made, but beating and pepper spraying a bunch of teenagers was a bit extreme.
Worst of all, the action of a few baton-happy cops reflects bad on the whole country: this has been the largest sporting event held in Canada so far, and now this incident will probably be one of the things people talk about.
Future someone: "Hey remember 2007 FIFA's U-20 in Canada?"
Future someone else: "Yeah! Where those Chileans players got beaten by the police?"
Future someone: "I wasn't thinking about that, but now that you mentioned it...Yeah, those dudes got
pwoned."
Sure, law and order is a good thing, but these were kids that had just lost one of the most important games in their lives. A little judgment and self control from the adults in charge (and the ones with the guns) would have been in order. BTW, I wasn't there, so I can't tell if the force applied was necessary, but it doesn't sound like it was.
More news:
- Chile presentará queja tras incidentes
- Gobierno chileno presentará queja formal ante autoridades canadienses
- Soccer team slugs it out with police
I'm thinking new tag line: "We're Canadians, eh? But we will beat the s**t out of you, if you step out of line..." I know, I have a bad sense of humor.
Taking advantage of the most needy?
Thursday, July 19, 2007
I have no problems with profits, but should we draw the line somewhere?
Early this week economists started talking about the economic decline in one of the most important cities in the country: Toronto (or the GTA, as is now called). The reports read that Torontonians' standard of
living are declining, because of the high unemployment rate and the strong Canadian dollar.
So what's a city to do? Well, why not increase legal gambling?
I don't get it: by promoting a vice to the ones that are probably addicted to the glamour of cheap casinos, the city is
planning to increase its revenues. I have no doubt it will work, but what about thinking outside box and come up with a better solution? It's not easy, but there has to be a better way to increase revenues than peddling cheap thrills to
gambling addicts.
Think about it: who is the most likely to gamble more when there is more gambling available? The one who never gambles, or a gambling addict? I'm not a gambling man, but I'd bet it's the gambling addict--I'd give anyone 10 odds to 1.
Why not decrease taxes to corporations so they set up shop in the city? Why not develop the waterfront area? Why not retrained the unemployed with useful skills?
Why not open beer and liquor stores 24 hours of the day, 7 days a week? Why not decrease the prices of cigarettes? Oh, wait. These two suggestion have a tinge of similarity to the current solution, yet no one is likely to support them. What's the difference? All those kids wanting to be cool would be able to afford coolness in a $3 dollar package--imagine all that revenue been wasted because of social responsibilities.
To quote is to not understand
Wednesday, July 18, 2007
Perhaps...
Reading maketh a full man; conference a ready man, and writing an exact man. And therefore if a man write little, he had need have a great memory; if he confer little, he had need have a present wit; and if read little, he had need have much cunning to seem to know that he doth not. Histories make men wise; poets, witty; the mathematics, subtle; natural philosophy, deep; moral, grave; logic and rhetoric, able to contend. Abuent studia in mores.
- Sir Francis Bacon, Studies
Indeed,
Abuent studia in mores*.
* Studies are transformed into behavior (The Broadview Reader, 3rd Edition).
Contex Matters
Compotation !=
Computation.
Who knew?
Football (soccer) score of 31 - 0...
Monday, July 16, 2007
I've never heard of such score until now:
I would think that after the first half, with a score 16 - 0, even the waterboy will get a change to play.
I wonder if this is the highest scoring FIFA-sanctioned soccer game in history?
Outlook 2GB email file limit [UPDATE]
Saturday, July 14, 2007
Read
this first, to get the context of this update entry.
After "repairing" the Outlook.pst file, I ran Outlook and it runs; however, there are large gaps in the data. For example, my contacts folder is useless now, as there are quite a few entries gone. I'm not sure what else is gone, but I have devised a different strategy.
Forget the Microsoft tools: I propose to export my important folders and then import them back into a clean Outlook.pst.
Of course, I won't export/import ALL my messages: I will be selective. For example, I'm not exporting/importing the 33,000+ spam emails I have--so I guess now my SpamBayes filter will have to be retrained with less messages.
I wonder who taught spammers to use email?
I'll post the results of my strategy. Right now the status bar of the exporting function says I have 3 hours to go.
[UPDATE]Exporting/importing the folders does work. So if you run into the same problem I did, I recommend you export/import your most important folders and forget the Microsoft tools--they work, but the mess left behind is not worth the effort.
Outlook 2GB email file limit
I receive way too many spam messages, so today my Outlook.pst file reached the 2GB limit set by Microsoft's developers. Needless to say, today is not a good day to be messing around with over-sized Outlook.pst files.
There was no indication this was the problem, as the error message is very vague. What do you think this means:
To me, it means anything but a size limit. And restarting Outlook does nothing. Of course, every time you reopen the application you get the same error message, so we can go on forever.
I took a guess that that was my problem, and it was. But how would anyone know? I looked at my Outlook.pst file and it was 1.95 GB, so I took a chance.
The worst part of it all is that you can't manually remove messages from Outlook: if you try, you get the vague error message I mentioned above.
To fix the problem you have to run a couple of tools--I should really say "try to fix," as there is no guarantee this will work.
First, you need to download and install
PST2GB. This tool "safely" removes messages from the original Outlook.pst file.
Second, after you have ran PST2GB, you need to run C:\Program Files\Common Files\System\Mapi\1033\SCANPST.EXE. This file "repairs" the mess the first program leaves in your mail file.
In summary, the fist program removes data and the second program tries to repair the damage, i.e., trying to tie pieces of the file into something that still looks like an Outlook.pst database.
Sure, you loose a few message here and there, and you can't control which messages, but do I really need to keep messages sent to me in 2002? I'm a data hoarder, what can I say.
This 2GB limit sucks. Well, not the limit, but reaching the limit.
Bart will be up and underchieving in no time
Is it really the end of
The Simpsons?
I don't want to believe it, but I think the end is near.
I give the series another 2 years. The social commentary is still there, but the quality of the writing has suffered.
What happened to all those Ivy league graduates? Where are these people going? Are they getting real jobs?
I'm looking forward to the movie.
BTW, the movies' site has a Flash application that lets you create a cartoon version of you. Do you see the resemble in mine?
GWT client + RSS Feed Reader + Code
Friday, July 13, 2007
I've received a few emails asking me for the code of a small program I wrote on February: a
GWT UI that reads multiple RSS feeds simultaneously.
The code can be found here:
sandoval.ca/gwt.
I think what most people are interested in is the actual RSS reader. That functionality can be found in the file
RSSServiceImpl.java. This is the important part of the code:
...
import org.gnu.stealthp.rsslib.RSSHandler;
import org.gnu.stealthp.rsslib.RSSItem;
import org.gnu.stealthp.rsslib.RSSParser;
...
RSSHandler handler = new RSSHandler();
URL rssURL = new URL(url);
RSSParser.parseXmlFile(rssURL, handler, false);
LinkedList list = handler.getRSSChannel().getItems();
int listSize = list.size();
listSize = (listSize > 10) ? 10 : listSize;
for (int i = 0; i < listSize; i++) {
...
RSSItem item = (RSSItem) list.get(i);
// Getting stuff out of the RSSItems
item.getLink();
item.getDescription();
item.getTitle();
theLink.toString();
}
...
Before writing it, I researched a couple of RSS Java APIs, but I liked
rsslib4j. I found it the quickest to code with.
BTW, if you download all the code and play around with the application, you'll find that digg's RSS feed doesn't work. I'm not sure why, but I think they have blocked my client; maybe they check which reader is parsing their XML file. If it works for you, let me know.
Canada's football has to be better than this
Monday, July 09, 2007
Ouch!
Canada's U-20 had the worst showing of any country in the 16 year's tournament history: not one win, and, worst of all, not one goal scored.
According to the Canadian media, the team has
been preparing for this tournament for 2 years. The question in everyone's mind is what happened?
Chile, Argentina, and Mexico are the top contenders. Brazil is always a possible winner, however, they classified 3rd to the round of 16.
It will be interesting to see if the youth and senior squads win their respective tournaments. Argentina, Mexico, and Brazil are always contenders for the cup--in any cup. And so these 3 teams are also battling it out in La Copa America tournament.
Great summer so far. Busy at times, but great soccer matches to distract the busy mind.
So Transformers rocks...
Sunday, July 08, 2007
Transformers is one of the best movies of the summer, yet some movie
critics are not digging it.
I'm not sure how anyone can not like this movie. It's like a boy's dream come true: having a super cool car as a pet, and, to top it all off, your cool car has other cool cars as friends; and your cool car's enemies are also super cool. You can't go wrong.
This movie is amazing. It is nothing like the cartoon, only a million times better. Yes, there are cheese parts, but nothing that a good 150 km/hr transformation can't make you forget. In other words, by the time Megatron has completed with its transformation, you've already forgotten the cheese line the cute girl said--really, I don't even remember any of the cheese lines (and I'm really trying to recall).
Forget the 2.5 hr GM commercial, the movie rocks. I do wonder, though, how a Toyota Corolla would like as a decepticon robot.
FIFA U-20: Chile 3 - Canada 0
Tuesday, July 03, 2007
Chile beat Canada 3 goals to 0. I wasn't expecting Canada to win, but I thought the Canadian team should have done better.
I don't blame the kids for such horrible showing. I think the coaches can do better by selecting better players and teaching them how to really play the game--kicking the ball with no purpose is not playing soccer.
It's unlikely Canada will win a game in this tournament, but I wouldn't take it too hard. And if I were a journalist, I wouldn't make it sound as if the world has ended. In one particular instant, a National Post
writer borderlines on exaggeration. He writes, "Canada's 3-0 loss brings entire nation crashing to earth." I don't even think many Canadians know that the FIFA U-20 is taking place in Canada, let alone know that Canada lost a game.
It will take a drastic change in the development of the sport in North America for national soccer to reach the level of Latin American or European countries, but to dismiss any effort put in by the poor mis-coached young players is mean. Of course, they were bad and must take responsibility for how bad they were, but they are not the only ones to blame. Why select them if they won't perform the way they should?