True to my nature ….

[and having some extra time on my hands], I picked up some research material to see the overall trend of things in 2009. Books, blogs, finance.yahoo.com and finance.google.com proved to be enough for me to paint a good picture [still with a lot of room for improvement] of what happened. Here I will only look at the reasons [atleast the ones which I feel are most relevant] for the failure of the financial markets.

Classical and Neo Classical Economic ….

theories got a wake up call finally. The new economic plans including stimulus are partially based on pure Keynesian teachings. The question that still remains to be answered is ‘Why did Capitalism fail ?’. Was it greed ? Was it plain simple stupidity on the parts of banks ? Was it the lack of regulation ? Maybe it was all this and more.

Lets look at capitalism for a minute. Capitalism states that all markets are efficient and information produced is available and shared by all people. In todays day and age this is a fair statement. There is enough data out there for rational investors to get out of bad positions or stay in the good ones. So if data is on the average easily accessible, what went wrong ? Read on.

The over reliance on normality based prediction models …

and the erronous data that they produced, in my opinion, is to blame. [DISCLAIMER: I HAVE AN ACTIVE INTEREST IN RISK MANAGEMENT].

Risk models are based on a normality distribution which does not take into account any unusual spikes. Most of these models are based on the axiom that future can be predicted by past. Bell curved calculations by necessity ignore black swans and thus risk managers supply measurements which dont consider these time bombs.

Somehow the failures of the risk models seems the same as getting caught counting cards in black jack. The risk modellers kept counting the deck [synonymous with looking at the past], made their money [synonymous with the great returns] before getting caught like Jim Sturgess’s character ‘Ben Campbell’ by Lawrence Fishburne’s character ‘Cole Williams’ in the movie ‘21′ [synonymous with the black swan events of 2007 onwards].

Whats the way ahead ?

I am not an expert on economic policy or even risk management therefore I dont have an answer but what the governments all over have done by introducing the stimulus is, in my opinion, a step in the right direction. Its a very Keynesian thing to do and while I suspect supporters of capitalism are having difficulty breathing, the markets in order to come back up need a crutch to hold on to and if that crutch comes in the form of a stimulus package …. so be it !

[If you want to read more about the long tail concept, click here].

I just happened to stumble upon…..

what I would call the most innovative expression of the ‘long tail’ in the world of social networks. The social network in question is catering to people who want to have affairs on the side while they are in relationships [yes you read that right - its a social network catering to people who cheat in relationships ].

All the moral issues apart, this has to be one of the most original and creative application of the ‘long tail’ concept. The networks web address is http://www.ashleymadison.com/ and has a very provocative tagline [take a look below]:

In true ‘customer-centric’ spirit, the web site……

guarantees an ‘affair’ and boasts over 4 million anonymous users.

This could be a sham…..

and even if it is, its a great illustration of how human ingenuity can find ways to make a buck.

Credit card regulations became stricter in 2009….[click here]

and forced credit cards to give more information to consumers. A lot of public talk was centering on how credit card companies cannot be trusted and that they intentionally did not give enough information to the consumers to keep them in the dark. Speaking from a personal experience, I saw a lot of my friends and family put credit card companies in the same league as crooks and murderers.

Olivia Chow, MP [NDP] also came out guns blazing against the credit card companies and wrote about a list of 11 ways credit card companies rip people off.

Why I think this campaign was started……

was to win back customer’s faith. The financial crisis last year [which continues unabated till now and if my accountant is to be believed will do so till the end of 2010] has left a lot of people in unhappier conditions. Times are tough and money is tight. The consumer spending train has been halted significantly. Credit card company’s like Visa, Mastercard and AMEX had a rough time during the last quarter of 2008 all the way to the first quarter of 2009. See attached comparison chart below:




but things are picking up again. The days ahead seem to be favorable for these companies to gain their profit margins and analyst have predicted strong EPS for VISA, MASTERCARD, DISCOVERY and AMEX.

People will spend freely once more…..

and therefore the credit card companies, in this case AMEX, have to ensure that any negative publicity that has been attached to their brand names and business models need to be neutralized. Their strategy includes repairing any damage done to their relationship with the spending population and in order to provide customers with the perceived value of ‘we care about you more than we care about the money you spend on us’, focussed marketing campaigns will need to be undertaken.

Other card companies should/will soon join in…..

with AMEX and start their own re-bonding efforts with the customers.

How much success they have is a question that is probably worthy of being studied with advanced statistical and behavioral financial and economics studies. It is my feeling that the new mind set that emerges, atleast for the next year or two, will be a lot more conservative when spending.

A word about the campaign design

- Color combination = A+. Very attractive and eye catchy.
- Using numbers to spell names = A+. Creativity at its best.
- Perceived message [by me as a customer] = C+. It seemed to me that I was being told that regardless of how much we care for you, we will still associate something as personal as your name with numbers because at the end of the day, we really associate you with what you spend on us - but we still proclaim to not care about your numbers. The message seems a little contradictory.

This has been one of my most involved projects - and even though the value of this project will not be realized for a while, it is nevertheless one of the most intellectually challanging thing I have done.

If anyone is interested in getting the source code from me, please let me know.

For the next week, I ll take a break and learn about the CAPM and other applied quantitative models for finance.

Collective intelligence - An architectural guide for scalability and efficiency

A good basic intelligence engine would consist of synchronous and asynchronous services. These services can be web based and/or have an offline set up.

Synchronous services service need to be fast because clients are waiting on the results of its processing.
In order to ensure proper scaling and high performance, synchronous services should be stateless i.e. the service instance should not keep track of any information during its round trips between the client and the intelligence engine. All the data needed for such services to successfully perform their tasks can be saved in a persistant data store or in a cache.
If there are multiple instances of such services running together, they should be behind a well configured load balancer.

Asynchronous services are typically in the back ground. Examples of this kind of service include the kind of data collecting web service which I created for an intelligence engine which recommended TV shows, movies, books, images, music and other media [the recommendation list was then delivered to either the users cell phone, a personalized web portal or a personal inbox on the users TV].

The asynchronous service that I created was stateless. It would accept the data that was passed onto it and go on to complete its intelligence task. As soon as it was done, another load of new data would then be handed to it, from a queuing system, and it would continue to chug along to do what it was supposed to do. Asynchronous services can be used for tasks which are longer and take more computation. Things like predicting user preferences, indexing existing data bases, calculating item ranks etc for individual users are examples of computational activities where asynchronous services could be used with greatest benefits.

The diagram attached shows a glimpse of the entire architectural layout. The request generated could come for a variety of media types e.g. video, music, blogs etc. The synchronous interfaces will directly contact the pre computed data sources to retrieve lists of potential ‘you might like these’ items. Such items are populated in the precomputed data cloud by the data crunching that is being handled by asynchronous services. A real time event, in our case the request, will carry new information on what is being asked for and will initiate the process of updating or creating new recommendations.

Creating or updating recommendations in real time

If the architecture shown here is implemented as an event driven SOA, it will allow real time update or creation of new recommendation items for the user. As soon as a new request is sent to the server, it will pass it onto the intelligence engines in the back ground. The next time the user interacts with the system, he should be able to see updated pre computed data for himself.

An example would be of amazon.com. As soon as a user engages in an event like searching for a book, a message is sent to the intelligence engine. The engine will then perform the relevent actions and the next time the user goes back to amazon.com, he will see that his recommendation list is different from the last time he logged in.
Similarly, if a user identifies a recommended item as being one he does not like, the intelligence engine is notified of the event. The recommended items list is re computed and presented again to the user. See attached screen shot from amazon’s recommendation list for a user :

I had been analyzing the merger for both Microsoft and Yahoo before they had reached a consensus in Jerry Yang’s days as CEO of Yahoo.

The only thing that seemed to make sense, from a classic economic point of view, was the large increase in the ‘economies of scale‘ for Microsoft and  Yahoo. The merger would allow both companies to enjoy efficiency gains that result due to increase in size.

This article was published in the New York Times after the merger had finalized and provided evidence that my sense was correct.

Enjoy !

I think this is a great move from Microsoft. This announcement could possibly have come as a result of Google announcing the release of Chrome OS and could very well be Microsoft’s effort at tit for tat play, but I doubt if Microsoft will not go through with this. With Google gaining more ground every day, Microsoft has no option but to live up to any hype it creates if it wants to get back its customer base and create new ones.

Why this move from Microsoft could be detrimental for Google

I love Google. I think Google is the ‘coolest’ software company in the world. They have broken Microsofts strong hold on the the productivity applications market with Google Docs. However, where I enjoy using Googles free services, I also enjoy, as much if not more, working with Office 2007. Office 2007 is a great improvement over Office’s older versions and is truly a phenomenal application suite.

Google, with Google Docs, in reality ‘commoditized‘ productivity software. Granted formatting in Google Docs was not always as visually pleasant as it is in Office desktop versions, the online nature of the service which lead to real time collaboration truly was useful.

Being someone myself who has to collaborate with team members all the time, Google Docs became the most useful tool for me. Never again was I suffer from version mismatches which can arise with just passing around soft copies of documents. However, when it came time to create a final document for presentation, I would invariably gravitate towards Office.

Now with Office 2010 being an online and a native desktop application [with the ability to synch between the two domains], I can see myself [and a host of other current Google Docs users] switching to using Office exclusively for my needs.

People might argue that Office 2010 will still need a subscription for access to more useful features, the fact that I can use only 1 provider for all my document editing needs will convince me to pay for the extra features.

Microsoft has shown great maturity

Microsoft has come out and embraced cloud computing whole heartedly. Infact they have found a way to integrate their core business [i.e. software selling] with a rivals competitive advantage [i.e. Google's ever growing emphasis on cloud computing] to create a new business model.

Good for consumers

This latest development in the world of productivity software is a definite boon for the end users. Google and Microsoft both have a brand name that will create ripples in the market whenever either one of them steps out of line and tries to chart a new path. Speculation about the success or failure of their projects will always be rife - and admittedly they will fail as much as they will succeed. However, the consumers will be the greatest winners in these cyber wars. Both competitors will probably keep trying to out do one another in their quest to enlarge their customer base, keep offering newer and better products for lower prices [or free of cost].

A speculation rife on the internet is that Office 2010 will only work with IE but this fact does not seem to dissuade me from using it. After all, we have to realise that Microsoft has had monopolistic tendencies and expecting them to not try to be one is, atleast for right now, unrealistic.

But a great productivity software for free from one of the biggest software companies in the world seems like a bargain too hard to miss.

Before reading on, please read the article posted at the link above.

I predict that google will take a heavy bounty from this ensuing war. For the longest time, things just dont seem like they are working in Microsoft’s favor. First the monopoly lawsuits, then the open source set backs followed by Apple’s resurgence and in the recent past, the bad image they picked up due to Vista’s ever increasing technical problems.

Google will get positive returns from this latest foray of theirs but it will take some time. Younger people will be more interested in looking at Google than those who are set in their microsoft ways and here is where Google can take a page out of Apples book and focus on some aggressive and intelligent marketing. Apple’s aid campaign made a lot of microsoftie’s convert to Apple. If I, who still uses Windows for most daily laptop related tasks, can go out and buy a Mac Book Air just because I found the Apple TV ads really entertaining [manipulative ], then believe me when I say a lot of other people would to aswell. 

Microsoft has its work cut out. Just by virtue of the fact that they will possibly be playing catch up to Google [and possibly even Oracle], they will always be behind the 8 ball. In these times of rapid technical changes, following in a competitors footsteps is not a recipe for success.

I have a soft spot for Microsoft and I just hope they dont become the next Yahoo.

Mr Padolny wrote this article in response to another article, published in the Times of London, which has questioned an MBA students ethics. The Times of London article, written by Philip Delves Broughton, openly blames the current crisis on the lack of proper ethical training of an MBA. According to Mr Broughton, the MBA is taught not to look at details but to try and implement every fancy new bit of business theory regardless of how relevant it is to the overall health of the company.

In his response, Mr Padolny has primarily agreed with Mr Broughton and has also presented  a solution to the problem. Mr Padolny talks about taking a more holistic approach towards teaching management skills to business students. He mentions Stanford, Yale, Ivey and Rotman as those business schools which have already adopted an integrative approach and he feels the philosphy of integrative thinking is what is needed for future MBA’s to be more adept at making moral management decisions keeping all details in sight.

This bodes well for future Rotman grads if they plan to travel south of the border after they graduate. The fact that Rotman is a known quantity and is mentioned along with the ‘crem del la creme’ of North American business schools should augur well for future grads. It should also give confidence to the current / prospective student body that choosing Rotman over other MBA schools in Canada is not a bad decision.

The full article can be read in the June 2009 issue of Harvard Business Review.

The DBA who saved the day

During the time I mention above, a senior DBA told me how to generate insert scripts. Suddenly the process of data migration became a formality. It became easy and I used those insertion scripts every chance I got.

However, I forgot how that script was generated till now. Recently I came across an article where such generation techniques are discussed. The article can be found at sqlservercentral.com and I hope it helps someone.

I know I ll be using it a lot for data migrations whenever and whereever.