Dynamics
|
|
Generational Dynamics |
| Forecasting America's Destiny ... and the World's | |
| HOME WEB LOG COUNTRY WIKI COMMENT FORUM DOWNLOADS ABOUT | |
In the companion article "Healthcare.gov -- The greatest software development disaster in history", we described from a journalistic point of view the Healthcare.gov disaster, the biggest IT disaster in history. In this article, we'll refer to that article as the "companion article."
We now know the reasons why it was a disaster: The Obama administration disbursed grants of $150-500 million per web site, where each web site project could have been done with a team of 5-10 people for a cost of $10-20 million. With so much money available to contractors, their objective was to spend the money, rather than develop the web site. As a result, there was massive fraud, including criminal fraud, and the Obama administration got what it deserved.
The purpose of this article is to provide information that can be useful to academic researchers, and to arrive at some "lessons learned," by expanding the discussion to cover all kinds of subversion, sabotage and fraud in software development projects.
I've depended heavily on the research published in the book The Dark Side Of Software Engineering: Evil on Computing Projects by Johann Rost And Robert L. Glass, published in 2011 by John Wiley and the IEEE Computer Society. This book contains a great deal of up to date academic research on subversion and sabotage, including numerous anecdotal descriptions of software development projects that crashed and burned. In this article, I've also included anecdotes from my own career, as well as observations that I hope will be useful to academic researches.
The Rost/Glass book makes it clear that sabotage of software development projects is actually quite common, with 20% of such projects sabotaged in one way or another in either minor or major ways. What I've found in my own experience is that about 5% of software development projects are completely dysfunctional, as I'll describe.
I particularly want to contribute to the research on this subject by relating examples from my own personal experience. Over the years, I've worked on hundreds of software projects for dozens of employers. I've also interviewed thousands of CEOs, CIOs, CFOs, and software developers and managers for hundreds of articles that I've written for CFO Magazine, InformationWeek Magazine, the Boston Globe, and others.
Finally, I want to relate the rise of subversion, sabotage and fraud to the rise of Generation-X, especially starting from the year 2000. It's now known that the financial crisis was caused by Generation-Xers who got Masters degrees in "Financial Engineering" in the 1990s, and developed a unique set of skills for defrauding investors, and then, in the 2000s, became bankers who used those skills to defraud thousands of (Boomer) investors by selling them knowingly fraudulent synthetic subprime-mortgage-backed securities.
I've tried to make this material as useful as possible to real people working on real technology projects. In particular, I've tried to focus on "lessons learned." If you're a manager working for almost any company, then you must have an IT department, and you may be exposed to the same kinds of fraud on a smaller basis that occurred in the huge Healthcare.gov case. Your company may not lose billions of dollars that the federal government lost on Healthcare.gov, but you might easily lose millions of dollars or at least hundreds of thousands of dollars. And it's important to understand that a disaster this size would not have occurred before the rise of Generation-X, which is part of the lessons to be learned.
My own background is both as a Senior Software Engineer and a technology journalist. The following is a very brief summary: Over the years, I've successfully developed hundreds of software applications for dozens of employers, from operating systems to compilers to web sites to complex enterprise-wide systems, working as both a consultant and an employee. (Resume: jxenakis.com/resume) I was Boston Bureau chief for InformationWeek magazine for two years, and Technology Editor for CFO Magazine (part time) for ten years, and I've interviewed hundreds of CEOs, CIOs, CFOs software developers and managers. (Examples: http://ww2.cfo.com/author/john-xenakis/).
Since I've depended on the Ross/Glass book for much material, I want to define some of the terms used in that book, and to describe how my own definitions differ.
The Ross/Glass book defines the term "subversive stakeholder" as follows:
"'Stakeholders' are people inside or outside the project who have any interest whatsoever in the software project and some influence over it. (That includes developers, project leads, architects, patrons, customers, consultants, and various user groups as well as managers outside the project).A 'subversive stakeholder' is a person who wants the project to fail — that is, a stakeholder who wants to sabotage, to disturb, or to destroy the project. Only people who act intentionally to the detriment of the project are considered 'subversive.'
Stakeholders who disturb the project due to incompetence or who are not aware of the consequences of their actions are NOT considered subversive in this survey."
Ross/Glass's research finds that over 20% of software projects have subversive behavior.
However, there are some important nuances missing from the above definition. In the Healthcare.gov web sites, the main problem wasn't stakeholders trying to sabotage or destroy the software project. It was the opposite -- stakeholders with disastrously failing software projects lying and cheating to cover up the failures, in order to keep the money flowing in. In my experience, this is quite common among dysfunctional projects.
The Ross/Glass book gives dozens of examples of sabotage of software development projects, but the authors are uncomfortable with addressing the people who expose those cases of sabotage - the whistleblowers.
Here's how the book describes whistleblowers:
"Whistleblowing (the act of exposing a wrongdoing in the hope of bringing it to a halt) doesn’t fit neatly into our book. The previous several chapters have all been about dark side acts, that is, the kinds of actions people take when they are stepping over into the dark side. People subvert, people lie, people hack, people steal information - all of these are dark side things people do.Whistleblowing, on the other hand, is in some ways the opposite of these acts. Whistleblowing is not an act of wrongdoing. Whistleblowing, as we have said above, is the act of exposing a wrongdoing. Good guys tend to whistle blow. Bad guys have a tendency to carry out dark side acts. Whistleblowers, in fact, may be blowing the whistle on the very kinds of dark side activities that we have been discussing in this book."
The book goes on to say that some people consider whistleblowers to be the bad guys, not the good guys. The distinctions can be extremely emotional, as exemplified by the case of Edward Snowden: Some people (including me) consider him to be a national traitor, while others consider him to be a national hero.
|
However, generalizations about whether whistleblowing is good or bad ignore what I call "the whistleblowing process."
In brief, the "Whistleblowing Process" is the process by which a "memo-writer" turns into a "whistleblower." This transition is, in my experience, a very important part of the fraud picture today.
As I've said, in my software engineering career I've worked on hundreds of software development projects, everything from small applications to major enterprise-wide systems, from embedded systems to compilers to web sites. I've worked as a lone developer, and I've worked in teams from two to ten people, using the "waterfall methodology" and the "agile methodology."
As part of my job, when I become aware of a problem, I write about the problem in a memo to my manager. The problem might be trivial or it might be serious, but either way my manager should know about it. I've probably written many thousands, perhaps tens of thousands, of such memos in my career. Writing a memo to your manager is not whistleblowing in any meaningful sense. I call it "memo-writing," because that's what it is.
But there's a "whistleblowing process" where memo-writing turns into whistleblowing. It occurs when the manager becomes abusive to the memo-writer or fires him.
As I said, I've written many thousands of these memos. There are only five or so of these memos that have gotten me into trouble. In most cases, the manager takes some action to resolve the problem, or thanks me and tells me he'll look into it. In these cases, life goes on, and nobody is hurt. But when the manager becomes so threatened by a simple memo, then there's a major problem.
Ross/Glass give several examples of whistleblowers. Here are some examples:
"In books on computing project failure, as we mentioned before, the subject of whistleblowing rarely comes up. For example, in the two books used as source material here, the term 'whistle - blowing' only was used on a couple of occasions. We've discussed reasons that probably suffice to explain why whistleblowing is so rare on computing projects. But there is one further reason, alarming in its implications, that is worth bringing up. Whistleblowers on these projects knew that trouble would follow them, perhaps immediately, if they blew the whistle.Earlier we mentioned the CONFIRM travel reservation system. In a chilling discussion of the desirability of having low-level employees speak up (they tend to be the ones who see problems coming the earliest), it was noted that 'some employees did complain about the technical problems — several of them paid for this with their jobs.'
Westpac is one of Australia's leading banks. Some years ago, Westpac embarked on an ill-begotten attempt to produce their own integrated banking information system, CS90, with the expectation that it would be so avant-garde that Westpac could sell it to other banks and make a great deal of money on it. To do that, they proposed to use concepts so advanced that most software specialists had no idea how to do them, and there was considerable animosity generated when they tried to do it anyway. ...
Craddock's version of the story is particularly interesting because he actually worked on CS90. ...
In one such episode, Craddock and his colleagues complained about the infeasibility of what they were being asked to do, and were told to 'shut up and get on with it.' (Craddock also said 'We were asked to believe at least six impossible things.')
On another occasion, Craddock had written a program to check for some program files that progress reports had indicated were complete; Craddock's program found that no such program files existed, and he was told to 'destroy the program' that uncovered the misrepresentation of what was happening."
I can identify completely with all aspects of this story. As I said, I've written many thousands of memos to my management describing some problem, and out of those thousands, there have been only five or so that have gotten me treated abusively or fired.
But the point isn't what happened to me. The point is how destructive it is to the project to ignore memo-writers. I've seen projects crash and burn because they ignored really obvious things. The Healthcare.gov project was a major humiliation to President Obama and wasted billions of dollars, because no one wanted to listen to the memo-writers. If the memo-writers had been heeded, then this disaster could have been avoided, and Obama would have had a successful implementation. In my experience and observation, I've seen millions of dollars wasted and projects crash and burn because warning memos were ignored.
Academic researchers should start to pay more attention to what I call the "whistleblowing process," where memo-writers are treated abusively, with the result that the project completely crashes. And technology managers should understand this process if they want their projects to succeed.
The Ross/Glass book considers the question about why whistleblowing is so rare in software development projects, even when the projects are headed towards failure.
The book gives several reasons why whistleblowing is so rare:
The Ross/Glass book gives several examples of abusive treatment. Perhaps the most dramatic was this one comes from this story from the wife of a whistleblower:
"Nineteen years ago, I married a man who was outgoing, secure, bold, and optimistic. Harry was self-confident, enthusiastic, and ambitious about his career. As a husband he was interested in and supportive of my activities. Later, as a father, he was involved in the lives of his five children and made every effort to spend as much time with them as possible.[After his boss started treating him abusively,] Harry became withdrawn, spending whatever hours he could in isolation, poring over his documents, compulsively reading and rereading every memo dealing with his work situation. When he and I did sit down to talk to each other, he could speak of nothing but what was happening at work and what his supervisors were doing to him, the pain and suffering he was going through evident in his shoulders and strained quality of his voice. ...
If his supervisors had set out with the intention of destroying his personality and ego as well as his career by subjecting him to threats, harassment, abuse, intimidation, and humiliation, they were totally successful. He who only wanted to do what was right was instead the victim of many wrongs."
This story is, of course, extreme in the aftereffects of the abusive treatment by incompetent managers. But it's not extreme in the sense that Harry's management purposely abused him with the objective of getting him to be quiet. When an incompetent manager's budget is at stake, then he won't hesitate to use fraud, abuse and firings to protect the flow of money, as we saw in the Healthcare.gov case.
Since I've personally worked on hundreds of software development applications for dozens of employers, and I've interviewed thousands of managers as a tech journalist writing hundreds of articles, I'm in a unique position to assess different kinds of software development projects and software development managers. I'm talking about differences in software development culture. I've developed some very strong conclusions, based on my experience.
First off, by "different kinds," I'm not referring to technology differences, such as an operating system project versus a web site project, or C# versus Java. Dysfunction can occur with any kind of technology, and in fact any kind of project, including political projects.
Second, I'm not making a distinction between projects that complete successfully and those that don't. A software development project might be canceled for any number of reasons -- marketing loses interest, the project misses development milestones, or a better idea comes along. But if a non-dysfunctional project is canceled, then everyone just moves to the next project, and life goes on.
The distinction that I'm making is to recognize "dysfunctional projects." Typically, these are failing projects where, in my observation, the management has no idea what it's doing, resulting in a dysfunctional team culture. In other words, this is about incompetent development managers, as opposed to incompetent developers (although incompetent managers may result in incompetent developers).
What's interesting is that I've seen specific commonalities among dysfunctional projects, and these commonalities can be used by corporate CEOs as early warning symptoms or signals of a dysfunctional project.
In some cases, the secrecy becomes delusional to the point of paranoia. The incompetent manager blames someone else for his own disastrous decisions, and when someone points that out, then that person is viewed as "out to get" the manager. This begins to cross the line into an actual mental health problem. Medical News Today
Let's focus on these signals/symptoms. As I've said, I've worked on hundreds of software development projects for dozens of employers in my career, and I've probably written tens of thousands of memos, many, many of which have warned of problems or questioned what's going on. The vast majority of these software development projects were NOT dysfunctional; they were almost always completed successful, and my memos were handled by management in a healthy fashion.
I asked whistleblower Dave whether he was screamed at when he was fired by UMass during the Mass Health Connector disaster. I quoted him as saying, "There was a big battle." I asked him whether that involved screaming, and he said it didn't, that it was all handled in a businesslike fashion. So I would conclude that the people at UMass were not paranoid and delusional. They were simply making a clear, unemotional, business decision that they didn't want anyone to know they were committing fraud.
However, I can't say the same thing in other cases.
In the companion article, I described how hostile and abusive CBS News and the Obama administration were towards Sharyl Attkisson, especially after she was the one who reported that only six people had signed up for insurance on Healthcare.gov launch day. According to Attkisson, Obama administration officials called her up to literally scream at her while she was working some stories. When Richard Nixon was president, he was often accused of being paranoid and delusional towards the press, but my observation is that President Obama and his administration are far more paranoid and delusional than Nixon ever was, believing that the press is out to get him -- not just Fox News, but also the NY Times and liberal news sites when they didn't toe the party line. Even left-wing reporters are calling the Obama administration the most hostile and abusive to journalists in history, especially after NY Times reporter James Risen and Fox News reporter James Rosen were both criminally investigated after publishing news stories not to Obama's liking. Washington's Blog and Washington Post
This kind of hysterical overreaction can indicate mental health problems if combined with more severe versions of the preceding two symptoms. These symptoms include "Delusions - beliefs that are not real; false personal beliefs that are not subject to reason or contradictory evidence," and "Anger - this emotional state may range from mild irritation, which most healthy individuals sometimes have, to fury and rage." I've definitely seen very extreme examples of these symptoms, but whether they're serious enough to be called paranoid schizophrenia is for someone else to decide. Medical News Today
I would warn any CEO or other corporate office that this kind of hysteria is a very dangerous sign, and could indicate that the software development project involved is very likely to be a total disaster.
|
The Ross/Glass book asks why whistleblowing is almost non-existent, and gives reasons why.
But that's the wrong question. In my experience, the programmers on a failing project are frequently aware that it's failing, and they tell their managers either informally or in writing, and they certainly discuss it with each other, and sometimes can't talk about anything else. That's the sign of a dysfunctional development effort, and the prelude to the whistleblowing process. Whether it turns into actual whistleblowing depends on the manager's reaction, and then the programmer's reaction to the manager's reaction.
In my experience as a "memo-writer" of thousands of memos, most managers -- the competent managers -- react to warning memos in a reasonably positive fashion, by either explaining why the warnings are wrong, or by taking corrective action. Even when a manager chooses to ignore the warning and go on, he usually gives some explanation for why he's doing that. I've even had a couple of managers who told me that upper management was aware of the problems, but wanted the project to continue anyway, for whatever reason. My reaction to that was that I had done my job by informing my manager, so I could go on ignoring the problem as well.
However, the incompetent managers are the ones who react by treating the memo-writer abusively or firing him. Even here there are levels, and a lot depends on the reaction of the memo-writer. The manager may tell the programmer to keep quiet and mind his own business, and the programmer may do so, or not. The manager may go further and treat the employee abusively by criticizing everything he does and micromanaging him -- and being micromanaged by an incompetent manager is so abusive that it may actually be an intentional tool to get the employee to resign.
Finally, the manager may fire the programmer, possibly with the intention of following a strategy that the Chinese describe as "Kill a chicken to scare the monkeys." When the monkeys (the other programmers) see that they're risking their jobs by saying anything, then they may be too frightened to say anything more to anyone. But then the dead chicken (the fired employee) is out of a job and is free to become a full-fledged whistleblower, if he so desires.
So the "Whistleblowing Process" is really quite complex, involving many steps and interactions, and it can be stopped at any number of points by action of the manager and the programmer. In my experience, that explains why there are so few whistleblowers -- because one person or another brings the process to an end.
Ancient stories about whistleblowers are certain to be extremely dramatic and tragic, since otherwise the stories wouldn't have lasted so many centuries. Furthermore after a few centuries, it's very hard to distinguish history from mythology. Any ancient myth is going to contain seeds of history, and any ancient historical story will, at the very least, emphasize and enhance the mythical heroics of the storyteller's side.
In the Broadway play Camelot, King Arthur is married to Queen Guinevere, and invites the French knight Sir Lancelot to join his Knights of the Round Table. Guinevere and Lancelot began having an affair. Everyone knew about it, including Arthur, but no one said anything until Arthur's illegitimate son Mordred became a whistleblower and exposed the affair. That set off a chain of events that led to war between England and France, and the play ends with the two countries at war.
This example shows another possible effect of a whistleblower's actions, similar to lighting a fuse connected to a bomb. In a software development project, everyone may be aware that it's headed for failure, but in a dysfunctional project, everyone remains silent to keep their jobs. Once a memo-writer or whistleblower makes it clear that a project is going to fail, then it sets off a chain of meetings and decisions that can cause the project to crash and burn earlier than the release date, when it would have crashed and burned anyway.
In fact, I'm not aware of any software development disasters that weren't completely obvious well in advance. The Healthcare.gov efforts were missing deadline after deadline almost a year before launch date, and anyone who pointed that out was silenced, abused or fired. When a project misses one deadline after another, you can be pretty sure that it's going to crash and burn.
Applying this observation to historical and mythical stories of ancient prophets, it's quite possible that the predictions of prophets can be understood in a secular way by assuming that they correctly observed and interpreted signs that others were ignoring, just as many project leaders ignore missed deadlines.
Generational Dynamics is a methodology for analyzing historical and current events, and using the analyzes to forecast certain types of future events. It has been extremely successful, and starting in 2005, I challenged anyone to find an analyst, politician, web site or journalist with a better record of predictive success than the Generational Dynamics web site. Several people have taken up that challenge, but none has succeeded, since no such web site or analyst exists.
The Biblical prophet Solomon had an intuitive understanding of generational theory, if we judge from Ecclesiastes. Biblical prophets Jeremiah and Ezekiel would have been aware of Ecclesiastes, and so may also have had an intuitive understanding of generational theory. Thus, for example, it may be possible to understand Ezekiel's prediction of the destruction of Jerusalem as being plausible in a completely secular interpretation, which would make them whistleblowers as well as prophets.
The mythical Cassandra is an archetype of prophetic whistleblowers. When her father, King Priam of Troy, ordered that the Trojan Horse be brought within the gates of Troy, Cassandra warned that the Trojan Horse contained Greek soldiers, and that bringing it inside would be a disaster. Cassandra was considered insane and she was ignored and reviled, something that has happened to me.
Surely Cassandra's prediction must have been pretty obvious. Why would the Greeks have built a giant wooden horse in the first place? Surely it was suspicious enough to merit a complete examination and inspection before bringing inside the walls of Troy, and then the Greek soldiers would have been discovered and Troy would not have been destroyed.
After the destruction of Troy, Cassandra was blamed for the disaster, and she was beaten and raped. She became the slave and mistress of the Greek king Agamemnon. She warned Agamemnon that if they return to Greece then his wife, Clytemnestra, will kill them both. Once again, the whistleblower is not believed. They return to Greece, and Clytemnestra kills them both.
The Cassandra story has many elements that are relevant to ancient prophets and contemporary whistleblowers alike. The predictions are perfectly obvious to anyone who even looks at the signs. The people who make the predictions are treated abusively or fired.
There's one more point to be made, about Cassandra and about prophets and whistleblowers in general. There's no pleasure in being right. Cassandra would have been devastated at the death of her father and the destruction of Troy, if only she had not been ignored.
This was captured in the lyrics of the Abba song 'Cassandra':
"While you are crying alone in your bed
Pity Cassandra that no one believed you
But then again you were lost from the start
...
So in the morning your ship will be sailing
Now that your father and sister are gone
There is no reason for you to linger
You're grieving deeply but still moving on
You know the future is casting a shadow
No one else sees it but you know your fate
Packing your bags, being slow and thorough
Knowing, though you're late, that ship is sure to wait
...
I watched the ship leaving harbor at sunrise
Sails almost slack in the cool morning rain
She stood on deck, just a tiny figure
Rigid and restrained, blue eyes filled with pain"
The same is true for me: Some of the saddest times in my career were to see a project crash and burn because an incompetent manager simply ignored the obvious signs of disaster, including multiple warnings from me. Every Cassandra is always "lost from the start."
Since 2003, I was what you might call a "public whistleblower" about the financial crisis, by tracing its growth and crises on my web site, GenerationalDynamics.com, which continues to this day. Of course I was too unimportant for anyone to pay attention. But even when I warned friends, they rarely paid attention to me either.
Most people don't realize this, but when Alan Greenspan was chairman of the Federal Reserve, he was aware of the stock market bubbles and the real estate bubble. I know this because I analyzed his speeches and interviews. But one of the reasons that few people know it is that he's apparently too embarrassed to admit it, even though he was right. And I can personally attest to the fact that being right only makes you hated. That may be why Greenspan avoids the topic.
So in this section, I want to tell the Greenspan story because it's highly relevant to understanding the role of whistleblowers in software development projects.
I remember very vividly the day in 2002 when I was eating lunch at the mall reading the Boston Globe, and I turned to the front page of the business section. This was at a time when many people were suffering the after-effects of tech bubble of the late 1990s and the Nasdaq crash in 2000, and were hoping for a full recovery (which of course means that were hoping for a return to the bubble). There was a huge graph showing the Dow Jones Industrial Average (DJIA) going back to the early 1900s. I was a math major at MIT when I was young, and since then had done a lot of work with exponential growth patterns and the Law of Mean Reversion. So when I looked at that graph, I knew immediately that there was a disaster coming.
I took one look at it and said, "Ohmigod, the stock market is in a bubble and is going to crash." (Yes, it was that obvious, just from the graph.) Later, I did the detailed analysis and verified that it was repeating the 1920s, exceeding historical exponential growth patterns.
It's incredible how naïve I was at the time. I thought that once this problem was identified, people would want to know about it and take immediate action. Instead, I was simply blown off by people I talked to. Some people got angry and abusive. Some people screamed at me, telling me I was full of crap.
Of course, this is exactly what happens to people in software development projects who say that the project is in trouble.
I set up my web site in 2002-2003 with the purpose of publishing analyses and predictions, so that people could judge for themselves whether the predictions were right or wrong.
By 2004, I was writing regularly that there was a large real estate bubble. But what few people realize today is that Alan Greenspan also knew that there was a real estate bubble. In a posting on my web site in October 2004 ( "Greenspan uses circular reasoning on housing bubble"), I analyzed an Alan Greenspan speech to show that he knew that there was a real estate bubble, but he thought it was OK because it gave households more money to spend -- by going into debt. I quoted him as saying:
"To be sure, some households are stretched to their limits. The persistently elevated bankruptcy rate remains a concern, as it indicates pockets of distress in the household sector. But the vast majority appear able to calibrate their borrowing and spending to minimize financial difficulties. Thus, short of a significant fall in overall household income or in home prices, debt servicing is unlikely to become destabilizing."
Greenspan's endorsement of the real estate bubble as being a good thing may be the reason why later he was too embarrassed to admit that he knew there was a real estate bubble.
This was in October 2004. In November, Greenspan gave a truly mind-blowing interview, which is retrospectively almost impossible to believe. ( "Greenspan-WSJ: Why did stock prices suddenly turn sharply upward in 1995?")
In the interview, he said that in 1996 he had known that there was a stock market bubble, but that he thought it was OK because technology -- the PC and the internet -- increased productivity so much that the bubble was justified. His reasoning was unbelievably awful, as you can see for yourself by reading the linked article in the last paragraph.
So by the end of 2004, it was clear that Alan Greenspan was aware of the tech and real estate bubbles, but he thought they were OK.
But then, early in 2005, Greenspan made an amazing U-turn in his reasoning. (See "Alan Greenspan reverses himself" from February 2005.) He completely repudiated the reasoning in his interview, just three months earlier, saying that the 1996 tech bubble wasn't OK after all, and perhaps he should have tried to do something about it as Fed chairman.
What I've always considered remarkable about this is that Greenspan completely reversed himself, but I've never seen or heard a word about this anywhere except on my web site.
All the economists and economics writers and journalists did at the time was constantly whine about how they didn't understand anything that Greenspan was saying. They blamed his convoluted sentences as making it impossible to understand him.
Actually, the pathetic economic journalists at the Wall Street Journal or CNBC or Bloomberg were so dumb or lazy, they didn't even care. When I listened to Greenspan giving a speech, I didn't understand everything either. But then I went to the Fed web site and read the speech in print. Even the most convoluted paragraph became clear after I read it three or four times. But the mainstream economics journalists were too dumb and lazy to do even that.
Finally, at the end of 2005, when Greenspan was leaving the Fed, he gave a speech indicating extreme alarm, and actually came close to predicting a stock market crash. I quoted him as saying in a speech:
"To some extent, those higher [stock and housing] values may be reflecting the increased flexibility and resilience of our economy. But what [investors] perceive as newly abundant liquidity can readily disappear. Any onset of increased investor caution elevates risk premiums and, as a consequence, lowers asset values and promotes the liquidation of the debt that supported higher asset prices. This is the reason that history has not dealt kindly with the aftermath of protracted periods of low risk premiums."
What he described here was exactly what happened 2-3 years later in the financial crisis.
At that time, Greenspan was leaving the Fed, and Ben Bernanke was taking his place. I've always liked Bernanke, because he seemed like a pleasant, honest man, a Charlie Brown who simply had no clue what was going on. In my October 2005 article, I compared him to Greenspan. ("Ben S. Bernanke: The man without agony") I quoted him as saying that the 1929 crash only occurred because of Fed policy errors:
"Without these policy blunders by the Federal Reserve, there is little reason to believe that the 1929 crash would have been followed by more than a moderate dip in U.S. economic activity."
In the years to come, Bernanke learned how completely wrong he was -- in fact, how completely and disastrously wrong his entire life's work in economics at Princeton University had been. In May 2008, I commented on a front page article on Bernanke's work at Princeton. ( "WSJ's page one story on Bernanke's Princeton 'Bubble Laboratory' is almost incoherent") The statements from Princeton's economists were so moronic I won't bother to quote them, but you can read the linked article to see for yourself.
But this goes to show, once again, how dumb and stupid the economists at the Wall Street Journal, Princeton University, and other places are. As I said a few paragraphs back, even Greenspan's speeches were too complicated for them.
I like to point out that mainstream economists have never explained why the tech bubble started in 1995, rather than in 1985 (the PC explosion) or 2005. Even today, 20 years later, they have no idea why it happened or why it ended. And they can't explain the credit or real estate bubbles of the 2000s decade, the "credit crunch" of 2007, and so forth.
Throughout the middle of the last decade, I was writing constantly about the housing bubble, and how it was going to crash. But mainstream financial analysts, economists and journalists would say, "Housing prices can't go down -- people have to live somewhere," and "Banks won't foreclose -- it's not in their interest to do so" and "These housing construction firms know what they're doing, and they wouldn't be building houses if it were just a bubble." It actually wasn't until 2010 that mainstream economists admitted that there HAD BEEN a housing bubble - something that was obvious five years earlier, but that they had been too stupid to see.
What was personally very painful to me is friends who were buying houses in the middle of the bubble. I begged them not to, but they pretty much all ignored me, and suffered greatly as a result. One of them actually blamed me later, as if I were responsible for her misfortune.
There is one more relevant story to be told here.
You'll recall the Bernie Madoff is the man who defrauded thousands of investors out of $65 billion.
Madoff himself has been quoted as saying, "In today’s regulatory environment, it's virtually impossible to violate rules. This is something that the public really doesn’t understand. But it's impossible for a violation to go undetected, certainly not for a considerable period of time." When Madoff said that, he had already been violating the rules and defrauding people for decades.
The ignored whistleblower was forensic accounting expert Harry Markopolos, who had repeatedly complained to the SEC that Bernie Madoff's claimed returns were mathematically impossible. He made five separate submissions to the SEC, in May 2000. October 2001. October, November, and December of 2005, and then again June 2007, and finally April 2008.
"It took me five minutes to know that it was a fraud. It took me another almost four hours of mathematical modeling to prove that it was a fraud. As we know, markets go up and down, and his only went up. He had very few down months. Only four percent of the months were down months. And that would be equivalent to a baseball player in the major leagues batting .960 for a year. Clearly impossible. You would suspect cheating immediately."
Markopolos said there were only two plausible explanations: either Madoff was using insider information to rack up the huge profits or he was running a giant Ponzi scheme. Both of those are illegal. Markopolos's repeated warnings were ignored by the SEC, and at times he feared for his life, until Madoff was exposed in December 2008.
Madoff's Ponzi scheme cost investors $50 billion dollars. He particularly targeted Jewish investors and Jewish organizations, suggesting that investors tend to be most gullible when dealing with people of the same religion or ethnicity. Many Jewish philanthropic organizations were put out of business after the fraud was exposed, and many individual investors committed suicide. Madoff succeeded in this scheme in full view of the public, despite whistleblowers.
The lessons to be learned here are the same as in the case of software development disasters. Warnings were ignored in the case of Healthcare.gov, and $5 billion were wasted, and the White House was thoroughly humiliated. I personally have observed tens of millions of dollars being lost after warnings were ignored and software development projects crashed and burned.
Whether it's a Ponzi scheme or software development, the warning signs are all there, well in advance, and ignoring them can lead to disaster.
|
Before beginning this discussion, I have to start by saying that I'm extremely sympathetic to Gen-Xers and Millennials, and I believe that many of their complaints about Boomers are valid. But on the other hand, I'm also extremely pissed off that Gen-Xers created the financial crisis, and that the Obama administration and other Gen-Xers refuse to prosecute Gen-X criminals, instead letting them go on to commit more crimes. Returning now to the first hand, I'm also extremely sympathetic to the fact that the entire Generation-X has become defined by a small collection of criminals, and that Gresham's Law has applied in the most vicious form: The bad have driven out the good.
Anyone who has worked in software development in the 1980s, 1990s and 2000s, as I have, must be aware of the enormous changes that occurred after 2000. Of course, the core work is the same -- writing, testing and debugging code. But the cultural changes are enormous, and that has to do with the rise of Generation-X.
Anyone can spend half an hour on the internet and verify that many Generation-Xers have a strong visceral hatred for Boomers. Just google the words "Generation-X hates Boomers" and you'll easily find plenty of evidence.
For example, a really nasty, sarcastic 2013 article by a Gen-Xer named Gene Marks gives five reasons why Boomers are the worst generation in history. Philadelphia Magazine, 13-Dec-2013.
Since I've heard the same kinds of things from other Gen-Xers, it's worthwhile listing his reasons. The article begins by saying, "Ten thousand [Boomers] are retiring every day. Good riddance," and gives these reasons:
This is why why many people call Gen-Xers as "nihilistic." In January 2008, I wrote "The nihilism and self-destructiveness of Generation X", and described how the deep hatred of Gen-Xers for Boomers can result in their own self-destruction.
We now know that it was bankers in Generation-X that caused the financial crisis. These were the kids who got Masters degrees in Financial Engineering in the 1990s. By 2004, they were 30-40 years old, and they were the bright geniuses that all the banks were competing to hire, because they knew how to make millions or even billions of dollars for a firm by slicing and dicing securities to increase their ratings in a way that was mathematically impossible.
They created tens of trillions of dollars of fraudulent synthetic subprime mortgage-backed securities and sold them to investors, bringing down our financial system. They didn't even feel guilty defrauding investors and bringing down the financial system, because they were like Gene Marks and considered the people they were defrauding to be in the worst generation in history, whose members they wanted to get rid of as quickly as possible.
This shows how Generation-X's generational hatred of Boomers can cause enormous catastrophes. What I've observed is that many Gen-Xers will make some decision simply because it's the opposite of what Boomers will do, and since Boomers are right more often than wrong, this means that Gen-Xers who use this approach will be wrong more often than they're right.
Gene Marks is just another internet blogger, and if you're not a Gen-Xer, you may think that his kind of vitriolic hatred and bigotry is rare. So let's quote some more famous people showing the same bigotry: Barack Obama and Clinton aide Paul Begala.
As Gen-Xer Barack Obama was beginning his presidential campaign in 2007, he was interviewed by the NY Times.
Some people claim that Obama is a Boomer, not a Gen-Xer, because he was born in 1961. In the article quoted below, Obama himself denies that he's a Boomer, calling himself a member of the "post-Boomer generation," which is Generation-X. It's true that the standard definition of Gen-X starts them in birth year 1963 or 1964. But from the point of view of generational theory, what matters is when a child is old enough to know what's going on, and is able to adopt generational attitudes and behaviors, usually around five years old. So in generational theory, the Boomers start from birth year 1942, and the Gen-Xers start from birth year 1959. So Obama is very definitely a Generation-Xer. NY Times, 21-Jan-2007
The following is from the NY Times article, and it's well worth reading:
"Shushing the Baby BoomersTHE time has come, Senator Barack Obama says, for the baby boomers to get over themselves.
In taking the first steps toward a presidential candidacy last week, Mr. Obama, who was born in 1961 and considers himself a member of the post-boomer generation, said Americans hungered for “a different kind of politics,” one that moved beyond the tired ideological battles of the 1960s. ... He is tieless and relaxed and oh so cool.
Mr. Obama calculates that Americans of all ages are sick of the feuding boomers and ready to turn to the generation that came of age after Vietnam, after the campus culture wars between freaks and straights, and after young people had given up on what überboomer Hillary Rodham Clinton (who made her own announcement on the Web yesterday) called in a 1969 commencement address a search for “a more immediate, ecstatic and penetrating mode of living.”
In his second book, “The Audacity of Hope,” Mr. Obama is critical of the style and the politics of the 60s, when the psyches of most of his potential rivals for the White House were formed. He writes that the politics of that era were highly personal, burrowing into every interaction between youth and authority and among peers. The battles moved to Washington in the 1990s and endure today, he says.
“In the back and forth between Clinton and Gingrich, and in the elections of 2000 and 2004,” he writes, “I sometimes felt as if I were watching the psychodrama of the baby boom generation -- a tale rooted in old grudges and revenge plots hatched on a handful of college campuses long ago -- played out on the national stage.” ...
BUT some say that after 14 years of personal and political self-indulgence in Washington and a grinding war, it’s time to say goodbye to the solipsistic generation.
“Thank you, here’s your gold watch, it’s time for the personal style and political framework of the 1960’s to get out of the way,” said Eric Liu, 38, a speechwriter and policy aide in the Clinton White House who now runs a mentoring program in Seattle. ...
Mr. Obama would be foolish to run solely as the anti-boomer, Mr. Lehane said, if for no other reason than that the baby boomers are the largest generation in American history, and they vote."
Imagine a politician calling some other demographic group, such as blacks or women, some of these names.
Suppose some politician referred used Obama's words and referred to "culture wars between freaks and whites" or Obama's words to criticize "the psychodrama of the feminists, a tale rooted in old grudges and revenge plots played out on the national stage." Those are all Obama's words applied to other demographic groups, and anyone would judge these Obama words to be bigoted and hate-filled. But he applies them to Boomers.
Let's take one more example, from Paul Begala who is a well-respected Democratic party political consultant, and was an adviser to President Bill Clinton. Like Obama, Begala is a Gen-Xer born in 1961, and in April, 2000, he wrote a widely publicized essay about his own hatred of Boomers. Esquire, April 2000
Here are some excerpts from Begala's essay:
"The Worst Generation by Paul BegalaI hate the Baby Boomers. They're the most self-centered, self-seeking, self- interested, self-absorbed, self-indulgent, self-aggrandizing generation in American history. As they enter late middle age, the Boomers still can't grow up. Guys who once dropped acid are now downing Viagra; women who once eschewed lipstick are now getting liposuction.
I know it's a sin to hate, so let me put it this way: If they were animals, they'd be a plague of locusts, devouring everything in their path and leaving but a wasteland. If they were plants, they'd be kudzu, choking off ever other living thing with their sheer mass. If they were artists, they'd be abstract expressionists, interested only in the emotions of that moment -- not in the lasting result of the creative process. If they were a baseball club, they'd be the Florida Marlins: prefab prima donnas who bought their way to prominence, then disbanded -- a temporary association but not a team. ...
In the long run, will it matter that one generation was so spectacularly selfish? Maybe not. In a great karmic irony, the Worst Generation may in turn be raising another great one. Having taught the children of the Baby Boomers off an on for five years now, at the University of Texas at Georgetown, I find them to be the opposite of everything I despise about their parents -- they are engaged in their communities, spending endless hours volunteering to build housing for the poor or to feed the homeless. They are concerned about their classmates, having calmed down the PC mania and replaced it with a sensible sensitivity to the feelings of others. They care about the future and are concerned about their grandparents. They are more responsible in their private lives and more engaged in our public life. I have no idea whether it's because of the Boomers or in spite of them. ...
But when that day comes, when they finally walk off the field -- or what's left of the field -- a few of us who've been trailing behind them will be doing a little dance of our own."
I'll give one more example. In one of my daily World View articles, I mentioned that Gen-X financial engineers were responsible for the financial crisis. A web site reader wrote the following:
"In the year 2000, the Baby Boom Generation was 36-54 years old and Generation X was 16-35 years old. Don't tell me that a bunch of 20-year-olds with unpaid internships used derivative mortgage-backed securities to bring down our financial system. Most people in business don't achieve any kind of power until they are in their mid-30s. If anyone is responsible for the corruption of our economic and political systems, it is the Baby Boomers.Granted, all this societal corruption began in the 90s. These were the Clinton/Gore years and Clinton/Gore policies led to our economy and society spinning out of control. Clinton is a baby boomer. Please stop dumping on GenX, we're going to have to clean up the mess when the free-love hippy baby boomers are gone."
Perhaps the above reader's comment really encapsulates the self-destructive nature of many Gen-Xers. Here's what I wrote in response:
"I guess you've never thought about high tech jobs. These were the kids who got Masters degrees in Financial Engineering in the 1990s. By 2004, they were 30-40 years old, and they were the bright geniuses that all the banks were competing to hire, because they knew how to make millions or even billions of dollars for a firm by slicing and dicing securities to increase their ratings in a way that was mathematically impossible. If they still had unpaid bills, their 6-7 digit salaries and bonuses would soon take care of them.And yes, I will tell you that those 30-40 year old Gen-X geniuses used derivative mortgage-backed securities to bring down our financial system.
And by the way, they didn't even feel guilty defrauding investors and bringing down the financial system, because they considered the people they were defrauding to be "free-love hippy baby boomers" that they wanted "gone" as quickly as possible."
Gen-Xer Vox Day (birth name Theodore Beale) is an extremely prolific and award-winning writer and blogger. Vox Populi blog
In 2012, he challenged some of my statements on Generation-X, and along the way provided some of the reasons why Gen-Xers have this visceral hatred of Boomers. Saying that "you have failed to understand something very significant with regards to the depths of Gen-X cynicism and fury," Day insisted that he and other Gen-Xers want to see Boomer financial managers "to go to jail, WE WANT THEM HANGED, DRAWN, AND QUARTERED."
As I responded, I've certainly heard this from Gen-Xers before, but that I don't believe that Gen-Xers really mean it. Sure, they'd like to see someone wave a magic wand and send the hated Boomers to jail, but that's not the way the world works. What I've seen is that Gen-Xers don't want the investigations to take place that would send the Boomers to jail because they would also implicate their Gen-X pals. The result is that no one goes to jail, and the evidence is overwhelming that this interpretation is true because, in fact, Gen-X prosecutors are not investigating banksters and making criminal referrals.
Day was kind enough to ask the Gen-X readers of his blog to tell whom they blame (Gen-Xers or Boomers) for the financial crisis. He was eminently fair in the way he posed the question, by quoting the following paragraph of mine in introducing the question:
"The evidence is overwhelming that Gen-Xers were the perpetrators of the crimes related to the financial crisis simply because it was only the Gen-X financial engineers who had the education and skills to create fraudulent residential mortgage backed collaterized debt obligations (RMB CDOs). And the evidence is overwhelming that they created and sold these fraudulent synthetic securities knowing full well that they were fraudulent. And the evidence is overwhelming that their Boomer bosses knew what was going on, but let it go on because they were making too much money. I've written about this evidence dozens of times."
I've written probably hundreds of times that Gen-Xers and Boomers have shared responsibility for the financial crisis, and this paragraph is a good summary. Gen-Xers provably must have been the perpetrators, because they were the experts on creating complex fraudulent synthetic securities, and their Boomer bosses would not even have understood how they worked. But it was also clear that there was massive fraud going on, and the Boomer bosses let it go on because they were making too much money. The Boomers and Gen-Xers had very different roles in the crisis, but they needed each other to make the crime work, just as a bank robber and the getaway car driver need each other to rob a bank the old-fashioned way.
Day's Gen-Xer audience overwhelmingly blamed the Boomer managers. Most people just answered with the single word "Boomer," but some added comments:
"Boomers are responsible, even if Xer's created the schemes, the Boomers could have stopped it, they had the authority and therefore the responsibility.The buck stops with the Boomer bosses, but it may well have started with the Gen X minions. Boomers and Jews are most accountable, and especially Jewish Boomers (can we call them say Joomers or something? it's almost like a neutron star of narcissism!) because they created the entire narcissistic cultural matrix in which these things became possible, even perhaps inevitable
Boomers as the bosses they are responsible by definition plus their collective decisions of the 70s and 80s to shut down the power grid expansion and legalize and then normalize fraud have left few options for the capable or ambitious.
Boomers. While Gen-X does have a share of blame, they have been taught via the mass media to worship the Boomers who saved them from the rigid conservatism of the "Greatest" generation. In many cases, they are following in their footsteps.
Boomers. They went from peace and free love to marketable securities and bad loans backed up by the blood, sweat, and tears of at least the next three generations without a thought or care.
Boomers are responsible by virtue of them being in charge. Mao Zedong never personally pulled a trigger, yet he is responsible for millions of deaths.
Boomers bar none. Who has been and still is in the majority of the leadership positions at banks, academia, the press, corporations and last but not least government? Boomer fucks that's who.
Given that most current CEOs and political leaders are boomers I believe the primary blame falls on them as they created the incentive system that led to this result."
There's an interesting historical comparison here, because this is exactly what's known as the "Nuremberg Defense," where the perpetrators of the Holocaust proclaimed their innocence, saying that they were only doing what their bosses wanted them to do. These defenses have been rejected in all cases based on the following "Nuremberg Principle":
"The fact that a person acted pursuant to order of his Government or of a superior does not relieve him from responsibility under international law, provided a moral choice was in fact possible to him."
But I think that Vox Day asked the wrong question. I knew that Gen-Xers would blame Boomers for the financial crisis. What I'd like to know is why Gen-Xer officials, like Timothy Geithner, refuse to investigate and prosecute suspected criminals. As I've written many times, the evidence that massive fraud was perpetrated by banksters in the 2004-2008 time frame and beyond is overwhelming, and yet there has not been a single investigation and criminal referral. When the S&L scandal occurred in the 1980s, there were thousands of criminal referrals. Even as late as 2000, the Enron scandal sent people to jail. But since the Gen-Xers reached middle management positions in the mid-2000s decade, prosecutors have adamantly refused to investigate and prosecute, even when it's obvious that massive fraud has occurred. I wish that Day had asked a question to get the bottom of that.
If the Boomers are really so guilty, then why are the Gen-Xers so goddam reluctant to investigate?
Actually, Day himself gave the reason in his e-mail message to me:
"The financial crisis is absolutely a Boomer affair. The reason GenX isn't concerned with playing the little regulatory game of the 1980s is because we believe the entire game is rigged, the prosecutions are fake, the punishments are wrist slaps, and the entire system has to be burned down in flames."
This is the Gen-X nihilism coming through. But his remark about the system being "rigged" allows me to make a connection.
I've written in the past about how Gen-Xers got the way they are. I trace Gen-X attitudes back to feminist policies, where they were raised in households where the only "father" was a string of men in their mothers' beds. They did not read feminist organization press releases, so they knew that their mothers were lying about domestic violence to drive their real fathers away and to get more money. As children, these Gen-Xers learned from the masters -- their mothers following feminist policies, destroying their families and their lives for money. Why not become a bankster criminal after being raised that way?
However, Day's remark about the system being "rigged" is that the divorce courts ARE rigged. As I've been writing since the early 1990s, feminist judges and social workers couldn't care less if the kids (Gen-Xers) were fed to a meat grinder, as long as they get their money. The whole divorce court system is rigged, and therefore Gen-Xers have concluded that all court systems are rigged.
When looked at that way, the Gen-Xer world view that led to the financial crisis makes perfect sense. If your mother lies to the court about phony domestic violence claims to get more money, and drive your father away so she can have a string of men in her bed, and if some of those men are abusive to her and to you, and if the court supports her to get their own money, even though the judge and social workers know the mother is lying, then of course you'll want to grow up to be a bankster, instead of a fireman or policeman.
Day wrote back to me as follows:
"This is absolutely correct! And it's not merely the courts or those who grew up in broken homes. We know we have been lied to about the most basic things since kindergarten by our teachers and our mothers. I think you are correct to suggest that Gen-X is in silent, but permanent revolt against the feminism that ruled our lives in our youth. The Boomer is shocked by Libor and other frauds, whereas the Gen-Xer shrugs and wonders when the manipulation of the commodity and stock markets will finally come out. We know all the courts and agencies and international organizations are rigged."
Day added the following:
"Seriously, I don't know if you can even conceive of the amount of pure, white-hot hate that many Xers conceal inside themselves. I was at the Game Developer's Conference when the Oklahoma City bombing [in 1996] was reported, and a lady reporter, a late Boomer, was aghast when a huge cheer went up at the news. When she asked one of the game designers, a famous one whose games you would recognize, why everyone had cheered, he told her 'basically, we're all terrorists at heart.'"
I don't believe that Gen-Xers are really pleased that Oklahoma City bombing occurred, but the momentary "huge cheers" do reflect the feelings of "pure, white-hot hate" that Gen-Xers have toward Boomers.
Day's reference to schoolteachers reminds of my experience with my son's schoolteachers in the 1990s. It was evident to me that these teachers (mostly women) considered boys to be nothing more than defective girls. These were the days of "the year of the woman," when racists like CNN's Jeffrey Toobin attacked a black man like Clarence Thomas for once having asked someone out on a date, while feminists like Susan Estrich sold themselves out as women and rape victims to save Bill Clinton, after he'd been credibly accused of violent serial rape. It's really no wonder that Gen-Xers, growing up in this putrid political and social climate, grew up to become banksters and bankster apologists. Reuters and Vox Populi blog
One thing that's become clear is that a lot of the fury that Gen-Xers feel toward Boomers is that most of them are children of divorce. In the 1980s, many mothers followed feminist policy to divorce the father, and then go into court a lie about domestic violence to get as much money as possible while shutting the father completely out of the child's life. Now I don't care what the feminists or the social workers say, but there's doubt in my mind that when any child grows up with no father in his life except for a string of men in his mother's bed, then that child is going to be screwed up and hate people in his father's generation -- in this case, the Boomers.
These articles on Healthcare.gov and fraud in software development project are a morality tale, as I've said before. Gen-X financial engineers wanted to destroy Boomers, and they ended up destroying themselves with the financial crisis. If you're a Gen-Xer who shares the views of the reader quoted above, then you need to re-evaluate much of your life, and how you make decisions.
|
It's well known that knowledge and use of technology is highly negatively correlated with age. In other words, Millennials are more comfortable with high tech gadgets than Gen-Xers, and Gen-Xers more comfortable than Boomers. Many Boomers won't even touch mobile phones.
This makes technology an obvious tool for young people who are contemptuous of older people and target them for fraud or otherwise. This is something that Millennials and Gen-Xers are very well aware of, but Boomers are almost completely oblivious to.
The use of high-tech synthetic securities was a big part of the financial crisis. Generation-X bankers used skills developed in the 1990s in "Financial Engineering" Masters Degree courses in the 2000s to create tens of trillions of dollars worth of defective synthetic subprime-mortgage backed securities, and knowingly defrauded investors with them. Their Boomer bosses had no idea for a long time what was going on, because they had no understanding of the technology behind the fraudulent synthetic securities. All they knew is that the fraudulent securities were making huge amounts of money. Yves Smith at the Naked Capitalism years ago posted some stories about how the bankers who had created the fraudulent securities had used methods of extortion to prevent their Boomer bosses from exposing the fraud. However, in view of the amount of money they were all making, I doubt that the Boomer bosses recall who was being screwed anyway. The Gen-X "financial engineers" didn't feel guilty about defrauding investors and bringing down the financial system, because they considered the people they were defrauding to be the Boomers whom they wanted to get rid of as quickly as possible.
Generation-X people in the computer industry have the same attitudes and behaviors, and many more opportunities to use technology, if there is a desire to defraud an older person. In my experience, Gen-Xers and Millennials are aware of these attitudes and behaviors, while Boomers are oblivious to them, making easy targets. Boomers, whether employees or managers, should be aware of what's going on for their own protection.
Let's look at some examples.
In August 2011, there were extremely destructive street riots in London. Young people destroyed and looted numerous stores, stealing large screen tv's and other items. Although looting was widespread, that wasn't the only motive. There were several BBC interviews of the looters who said something to the effect, "I did it because the police couldn't do anything about it. I just wanted to prove that we could do anything we wanted, and the police couldn't stop us."
One reason that the police couldn't do anything to stop the looters, at least for a while, is because the young people were using Blackberry Messenger as a private communication system to send riot locations to one another. The police were unable to monitor these communications, although after the riots ended, Blackberry maker Research In Motion cooperated with police to gain access to the private messages, for the purposes of prosecution. BBC 8-Aug-11 and Guardian 9-Aug-11
Also in August 2011, the mayor of Philadelphia was forced to impose citywide curfews because of a series of horrific attacks that were dubbed by the media as "flash mob" attacks, because word of them was spread by social media. The location would be posted on social media and young people would show up, beat the crap out of people, and rob stores. CNN (12-Aug-2011) and Tech President (12-Aug-2011)
In memo-writing or whistleblowing situations in technology, Boomers should be aware how potentially explosive the situation is when a Boomer tells a Gen-Xer that the latter has a problem. If you've worked in technology most of your life, then you're going to know a lot more about many things than a Gen-Xer whose been around 20 fewer years than you have.
I've personally seen numerous examples of the following: Many Gen-Xers become FURIOUS at a Boomer who knows more about technology than they do. This is almost incomprehensible to Boomers in technology, because the Boomer culture is of sharing with one another. There's a give and take in a conversation, where each person contributes knowledge and information, and they can agree on a common plan.
But that kind of sharing about technology is alien to much of the Gen-X culture. Here's how it's described in the Simple Programmer blog:
"Software developers are jerks. ...You don’t hear this much, because it isn’t really nice to say. ...
The truth is not everyone has your best interests at heart. The truth is that a large number of people would like nothing more than to see you fail; to prove once and for all that they are smarter than you.
Chances are if you are doing something unique or you propose a new idea, you’ll have more critics than supporters. I don’t say this to be mean or to discourage you -- I actually have the opposite intent. Rather, I say it so that you can be prepared and not think it has anything to do with your personally. Hopefully, I can help you grow the kind of thick skin you are going to need if you are going to succeed as a software developer. ...
The problem is that most software developers, male or female, aren’t really ready for the nastiness they are about to encounter when they start writing code as a career. Worse yet, much of this nastiness is disguised in a very passive aggressive manner, so targets of this ire aren’t even aware of it -- at first. ...
Where does this vileness come from?
We work in a sort of strange field where intelligence and ability are highly prized, but some of these same qualities made some of us victims of aggression and abuse earlier in life.
This tends to result in a culture in which many of the participants are constantly trying to prove themselves and evaluating themselves against others.
To put to plainly, it means there are lots of sensitive and bloated egos floating around. ...
This same kind of mentality also tends to foster cynical thinking and an outright rejection of any idea that doesn’t self-originate."
My experience endorses this view in the strongest possible terms. The Gen-X software development culture is not like Boomer culture. Based on what I've observed, the Gen-X software development culture is vicious. As the "Simple Programmer" suggests, Gen-X programmers are vile to each other, as well as to Boomers.
The same is also true in the banking industry, according to Sam Dogen at the Financial Samarai blog:
"[E]verybody knows that people in banking often work 70 hour+ weeks and get their face kicked in all the time. The pressure to produce is immense. If you don’t produce, you get shown the door."
The point to be understood is that Gen-Xers treat each other with the same hatred and contempt that they treat Boomers.
Age discrimination in hiring of technology employees is widespread and overt. I saw one job description for a programmer from the Portsmouth NH based recruiter Global Technical Talent. The description contained the following in large bold type:
"CULTURE is HUGE for them. They want the demographic of the person to be a good fit with the rest of the work environment--big open office space, collaborative team environment, young mentality!!"
I don't see how GTT could be more open that it's practicing illegal age discrimination, but if no one gets prosecuted for banking fraud these days, then no one is going to be prosecuted for age discrimination.
In one personal experience, I interviewed for a contracting job at Bank of America in Boston. The Gen-X group lead programmer was asking me a series of technical questions, and one of them had to do with the "Model-View-Controller (MVC)" programming pattern. I answered the question, and he corrected me, but his correction was wrong. I didn't even correct him, since I knew that would be disastrous, but I did ask him a question to clarify what he said. He stumbled in his clarification, and I could tell from the expression on his face that the job interview was essentially over. As I was leaving, he said, "We're very impressed with what you've accomplished." As it turns out, this is Gen-X code for "Screw you, Boomer. Now get the hell out of here."
Boomers are so accustomed relentless hostility from Gen-Xers that it's easy to fall prey to manipulative compliments. Using a compliment is an easy technique for practicing age discrimination. In darker contexts, it might be a someone giving you a compliment hoping that it will lead to defrauding you out of your life savings, or it might be a girl hoping for a one-night stand, pregnancy, and 20 years of collecting child support from you. I have two friends who were defrauded out of a great deal of money by Gen-Xers. I have a friend who was targeted by a girl from South America, who ended up taking the child back home, and still received her child support payments through the Massachusetts courts. Feminists organizations strongly support these scams because they're very lucrative for women's organizations, and family court judges often get kickbacks from the child support payments.
Age discrimination is illegal, but widely practiced with impunity. The same is true in the banking industry. Here's how you should modify your resume, according to Sam Dogen at the Financial Samarai blog:
The above advice is pretty dark, but what I've observed is even darker than the above suggests. I've seen situations where Gen-Xer managers made software development decisions that didn't even make any sense, but they were the opposite of what had been recommended by a Boomer, by me on one case and another Boomer in the other case. In both cases, the decisions were disastrously wrong, as events later proved.
Dogen's advice is to dumb down your resume, but similar advice also applies to any Boomer employee working for a Gen-X manager. If you want to be OK, then you have to follow similar advice and pretend every day to be dumb, or at least as dumb as your boss. Otherwise your Gen-X boss will fire you simply because you an older person, a Boomer who knows more than they do.
I call Dogen's advice "glorifying stupidity." You pretend to be stupid so your boss's feelings won't be hurt, and so he won't fire you. I wouldn't blame any Boomer from doing this to keep his job, but for me to try this would be soul-destroying.
Here's what I wrote in an article, in response to a comment from a web site reader:
"I'm sorry that I hurt your feelings, but what you and other Gen-Xers don't understand is that, as bad as Boomers are, Gen-Xers are worse. You think you'll be cleaning up after our mistakes, but you'll actually be making one huge blunder after another. Motivated by fury and anger at Boomers for doing nothing, you'll rush in to "do something", and the things you do will be disastrous -- lead to world war, lead to financial disaster. Your generation's utter contempt for everything that came before you, and your recklessly eager willingness to destroy it, will backfire on you and lead you to desperation and self-destruction. We're already seeing that with the disastrous results of the "financial engineering" that was implemented by Xers under the nose of Boomers. If you even survive the next 10 years, you'll come out of it bitter and angry. And it won't be the Boomers who'll be blamed or remembered for these disasters. You'll be blamed."
However, Gen-Xers should understand that no matter how much harm they do to Boomers in revenge, they will do far more damage to themselves, and it will be deserved.
I like to think of software development so far in the 2000s as having two bookends. In the year 2000, the first bookend was solving the Y2K problem -- the greatest IT success in world history. Today, the second bookend is Healthcare.gov, the greatest IT disaster in world history. It's no coincidence, in my opinion, that the trip from the first bookend to the second encompassed the rise of Generation-X.
A recent article on the site for Modis, an IT recruiting company, is titled "Top Tech Fails of All Time," listing the "most epic, top tech fails of all time." Modis.com
According to the article, the biggest "tech fail" of all time was:
"The Y2K BugRemember back in the tail end of 1999, when the digital sky was falling and computerize Armageddon wiped humanity off the face of the planet? No? That’s because it didn’t happen, though the frenzied global chaos in the year leading up the big jump to 2000 was probably worse than the minor aftermath that followed. The panic hinged on the fact the world’s computers were coded to read dates as two digits (1994 would be 94, for example), so the big question of “what would happen” when all of our computers flipped to “00” sparked terror over a worldwide computer meltdown. It did cause a few minor issues around the globe, but nothing on the grand scale that everyone expected."
This is one of the dumbest things I've ever read. Y2K was actually one of the most successful projects in history, especially in view of its global nature. However, the project was implemented by Boomers and Silents, and Gen-Xers who say things like the above are really expressing their contempt for older generations.
The Y2K problem was known in the 1970s, but started getting press attention in the early 1990s. As a tech journalist, I wrote numerous articles about it myself. It was clear that millions of application programs were going to stop working on 1/1/2000, especially IBM mainframe programs written in Cobol and PL/I, and that the vast majority of organizations in the world would be affected. By 1995, every institution was hiring or training teams of programmers to perform "remediation" -- modify all the company's applications to correctly handle 4-digit years. By the beginning of 1999, these teams were in a frenzy to get done. One by one, they completed their work during 1999, usually early in 1999, so that they were ready for 1/1/2000.
And they were successful! Far from being a "tech fail," this was the most successful tech project in history. It involved many thousands of programmers worldwide, and cost hundreds of millions of dollars. And it was managed and implemented by members of the Silent and Boomer generations.
This Modis.com article was personally offensive to me and a lot of other people too, judging from the comments posted to the article:
I worked for a telecommunications giant during that timeframe and we absolutely had a bunch of COBOL programmers brought in to sift through code to ensure that customer bills, service, and other issues were not impacted. We patched our Microsoft servers, our desktops, mainframes, databases, and the list goes on. Y2k is actually one of the epic "wins" in the last few decades for technology. I certainly don't want to downplay microscopic circuits, storage growing by 1000x, computing power increasing exponentially, etc, but to be naive enough to put y2k as a failure is completely repulsive.
Perhaps you should find a better blogger. That was utter nonsense. ... Actually, the biggest fail I've seen today is this blog entry. Fortunately, it's early in the day.
So, what reason does the modis.com article give for calling Y2K the biggest "tech fail" of all time? Because it succeeded!! It failed because it succeeded! That's how screwed up this writer is, and his attitude is shared by many Gen-Xers.
The biggest "tech fail" of all time was obviously Healthcare.gov, but that isn't even mentioned in the Modis.com article.
Since I've become particularly cynical in recent years, my assumption is that the bloggers and managers at Modis.com are well aware that they're lying, but they believe that they'll attract more young recruits and fewer older recruits if they lie about Y2K. By purposely lying and offending Boomers, they'll be saved from performing age discrimination later.
|
Programmer skills appear to be highly relevant to software development subversion, sabotage and fraud.
When I wrote my original article on Healthcare.gov, "1-Dec-2013 World View -- Obamacare: 500M lines of code, $500M, only 60% completed", I said that 500 million lines of code was impossible, and I suggested that it required 1,000 monkeys sitting at computers, typing random code, charging $200 per hour.
Well, there weren't monkeys involved, but there might as well have been. We now know that when the Obama administration granted $150-500 million per web site to develop a $10 million web site, then the objective of the contracting firms was to spend the money, rather than develop the web site. A large part of the problem is that apparently each contracting firm hired hundreds of monkeys ... errrrrrr ... programmers to develop the web site, and many of these programmers lacked even basic software engineering skills.
So in this section I want to address the software engineering skills issue.
In addition to having worked on software development projects with hundreds of different teams in my career, I've also taught a number of beginning and advanced programming courses in different languages, including assembler language, C, C++ and Java. So I have seen programmers struggle with coding problems, on the job and in class, and I've developed some fairly definite opinions on the issue of software engineering skills.
The bottom line: Many software programs contain code that requires skills and abstract thinking beyond the capability of even experienced programmers working on that very software.
That doesn't mean that the entire system is like that. A large software system may have hundreds of modules, and perhaps only a few of the modules cause the skill problems we're describing. In a web site or a desktop application, for example, many programmers do well on the user interface because the components are standard, and there's immediate visual feedback to all code that the programmer adds or modifies. But if that same programmer then tries to add features or fix bugs to a heavily threaded back end, he may cause additional bugs because he doesn't have the skills to understand what the software is really doing.
This doesn't have to be a problem, as long as there's a team of programmers working together. Different modules may require different programming skills, and so the coding tasks can be split up along skill lines. If an individual is having trouble with a particular module, he can ask for help from another team member. The Agile development methodology and "pair programming" can be useful in these situations. Every individual does what he's good at, and the whole team gets the job done.
Problems of fraud and subversion arise when a programmer works on a module that requires additional skills, and adds or modifies code that contain additional problems, sometimes triggering a disaster.
In my personal experience, the worst of these situations occur among Gen-Xers, who have a more competitive and subversive culture, while (in my experience) the Boomer programmer culture is much more cooperative and mutually helpful.
The third of the three situations is the one that leads to memo-writing and whistleblowing. When an incompentent lead programmer or manager starts forcing incompetent implementation decisions on competent programmers, disaster can only result.
It's a shame, because these problems can be easily resolved when recognized, by helping someone overcome his ego. But if they're not recognized, then there will be problems.
So let's look at some skill areas that cause problems:
When I was teaching beginning programming courses in various programming languages, I was surprised that I found the same thing would happen in all of the languages: Students could understand operations with numeric variables, such as "a=5; b=6; c=a+b;". They could also grasp the use of string variables, such as "s1='uvw'; s2=s1+'xyz';". That is, students could understand the value of a variable being a number or a string, and string concatenation.
But when I moved to the concept of the value of a variable being a pointer to another variable, I always lost about 1/3 of the class. That is, a statement like "p=addr(x); p->y = p->y + 1;" was too abstract to be understood by many students, even students with prior programming experience.
Once when I was working on a programming project, I was asked to help another programmer who was getting program aborts. I looked at his code, and noticed the following lines:
"if (p==null) {error=5;} p->depth=0;"
In other words, the person who had written this code had known enough to check for a null pointer, but then used the pointer anyway, getting protection violations when the pointer was null. When I explained this to the programmer who was having the problem, he nodded his head, but he clearly didn't understand what a null pointer was. (And that there's a difference between a null pointer, a null character, and a null statement.)
Another problem is that some programmers have no idea of what the difference is between automatic stack storage versus static storage. If such a programmer runs into memory issues, such as memory fragmentation that can occur in any language, they're totally lost. I've had the experience of trying to help a programmer with memory problems, and be completely confounded by the stack versus static storage.
Interpretive languages do a great deal to shield the programmer from having to understand anything about memory, but even in Java or C#, you can have serious memory fragmentation issues.
Depending on the language, some programmers are unable to understand the differences between value and reference variables. In Java and C#, many programmers don't understand issues around boxing and unboxing variables. Also, when comparing strings, some programmers will never understand why "s1==s2" sometimes works as expected and sometimes doesn't.
In advanced programming courses, where I described in class how to implement a linked list, sorting a linked list, adding a new element to a linked list, or removing an element, even programmers who had years of programming experience were simply unable to understand this example, or solve the homework problems related to it. Languages like Java, C# and Python get around these problems by providing ready-made classes for things like lists, maps and sets, which any programmer can use without understanding the underlying implementation, but they can still get into performance problems if they don't understand, say, the difference between sorted and unsorted sets.
The reason that I'm giving these examples is because they shouldn't be problems to a programming team. If the members of a team are able to cooperate and help each other, and if individual members feel safe asking for help, then none of these issues should be a problem. But if the team members are out to screw each other, as happens very often these days, then these issues become weapons in a brutal, bloody game of one-upmanship.
In my experience, few programmers have any idea what's going on in a heavily threaded program. I was working on a software system that ran dozens of network communication jobs simultaneously in different threads. The program got huge numbers of exceptions, but each time one of the threads aborted because of an exception, the system would simply log the error and restart the job.
When I looked at the code, I found many problems, such as this one, which removes a request from the queue:
while ((pending_requests.size() > 0) && (!isStopRequested())) {
synchronized (pending_requests) {
request = pending_requests.remove(0);
request.setStatus("executing");
}
...}
This code had been written years before, and a number of things like this had been ignored by multiple programmers. The programmer who wrote this code segment obviously had been told in some programming class to use the "synchronized" keyword to synchronize access to pending_requests, but who had no idea what this meant. There are two references to pending_requests, but only one is inside the "synchronized" block. Thus, it's possible that pending_requests.size() equals 1, but then that item has already been removed by another thread by the time remove(0) is executed. In that case, the remove operation would get an IndexOutOfBoundsException. That would explain the numerous exceptions.
I know from a number of such experiences over the years that this kind of situation goes beyond the capabilities of perhaps 90% of programmers, even programmers who do well at other things, and goes well beyond the level of abstraction required for pointers and data structures. And yet, the bugs that can arise are very insidious, creating race conditions that can do anything from making the whole system unstable to destroying database data.
Once again, this needn't be a problem in a cooperative and mutually helpful team, where people feel free to ask for help, and subtle sections of code can be reviewed by other team members. But all too often, thread synchronization becomes just another weapon in the war of one-upmanship, and the bugs accumulate.
"Big data" is a subject under a lot of discussion these days. Most people are aware that understanding of big data issues requires understanding such things as whether P=NP, or the difference in performance between adding an element to an unsored set versus a sorted set.
However, "big code" issues are less well understood. There are several different aspects.
A very large software system with many modules presents special problems for a programmer who's just joining a team. A change made to one module may have effects on the operation of another totally unrelated module. It takes a while for someone to become sufficiently familiar with the entire system to avoid problems of this type.
Y2K is another kind of large code problem. The 2-digit year problem is very concrete, unlike problems involving data structures or process threads, and so to a poorly skilled programmer the Y2K problem looks easy. And it would be easy if it were only necessary to fix one line of code. But in the 1990s, programmers were looking at billions of lines of code to be fixed, and that "big code" issue transforms a simple concrete problem into a massive problem requiring development of fairly sophisticated automated tools for remediation and testing.
Another example occurred on a project I worked on, where there were three reports that had different purposes and formats, but had to give consistent results, but there were subtle differences in the values in the reports. Each of the reports was generated by a 300-line SQL statement, joining numerous tables in various ways, each with hundreds of lines of post-processing code. Just as the three reports were subtly different, the three SQL statements were also subtly different, with WHERE clauses specifying time intervals in different ways.
The SQL code and the corresponding post-processing code were impossible to understand by themselves. In order to identify the differences, it was necessary set up a test framework where I could easily make minor changes to each SQL statement and see what effect it had on the result set and the post-processing code. It took a while, but I finally found exactly what was causing the problem.
What this example has in common with the Y2K example is that both seem to be simple, concrete problems, but because of the number of lines of code, each becomes a more substantial development issue because it requires additional software development -- a remediation tool and a testing framework -- to solve the original problem.
Another example is translating source code from one language to another. In the mid-1990s, I had to translate a large C program to C++, and also change the Microsoft C windows API to the Microsoft MFC API. I developed a script that could automate 90% of the translation work, and then editor macros could automate other pieces of the work. The amount of code that had to be manually changed was minimized.
There's a whole process that the programmer has to go through to make sure that these additional tools and frameworks will not introduce new bugs. However, that would take another 1500 words to describe, and this section of the article has gotten long enough. But once again, it's easy to see how a cooperative team of programmers can work on these problems, with the senior programmers specing out the approach, and leave it to the new team members to implement the specs. This seems obvious, but I've rarely seen these concepts discussed or described.
A final example of a different kind: An eCommerce web site was taking 10-15 minutes to produce certain kinds of parametrized reports. The online user could specify the types of data, and whether to summarize by day, week or month. The original programmer had implemented the back end with multiple SQL calls to the Oracle data base, and execution times got longer and longer as the number of days or weeks increased.
The fix was to write a module that would take the report parameters and generate an SQL statement that would gather all the data in one call. In some cases, the generated SQL statement was hundreds of lines long, something that Oracle handles very well. The performance time was reduced to a few seconds.
The final issue we'll mention briefly is security issues. This is important for any web site implementation, or for any application that communicates of the internet.
One solution to this skill problem is code reviews, where one programmer reads the code of another. There are many problems with code reviews, mainly because they're so time consuming, and so they can substantially decrease the productivity of a team.
Understanding programming skills is a way of determining where code reviews are needed. User interface code usually does not need to reviewed, since you have immediate visual feedback on whether the code works correctly. On the other hand, any code that updates the database has to be correct, and may need to be reviewed. Any code that touches on the skill issues that we've listed above may need a code review, but even here, the code review can be limited to specific areas in the code.
|
Systems integration is one of the least understood aspect of software engineering, and quite possibly the cause of most large system development disasters.
It's like buying a television set in the United States, taking it to Europe, and expecting to be able to just plug it in and watch TV there. It's not going to happen.
There's an old joke: The first 95% of system development takes the first 95% of the time, and the last 5% of system development takes the other 95% of the time. That "last 5%," which everyone things will be a breeze, often is the root cause of a software development disaster.
As I described in my companion article in the section on the disastrous Maryland Health Connection Obamacare exchange, an auditors' report months for the launch said that Maryland might build all of the components of its health-insurance exchange and then put them together and find out they do not work, and that the state also did not appear to be leaving itself with enough time to "complete, verify and test all system components before go-live."
My personal belief is that systems integration is the biggest cause of software development failure.
It works as follows: An incompetent manager breaks the software development up into functional areas (or use cases), and then assigns each of the functional areas to a different programmer to implement as a component. He may leave some time at the end for integration, but as components slip, there is no time for at the end for end-to-end integration and testing, and so on the launch date it's the TV set analogy at the beginning of this section -- you can try to plug it in, but nothing works.
For academic researchers, I suggesting reading the paper How Complex Systems Fail, Richard I. Cook, MD, Univ Chicago (PDF). Although this paper is not about systems integration, the observations about how minor unit problems combine in unexpected ways to cause major system disaster are very instructive.
In the sections on personal anecdotes at the end of this article, there are examples of major systems integration failures that are, I believe, extremely instructive.
From the point of view of developing software engineering projects, there was something quite significant in the development of the Obamacare web sites:
Deloitte Consulting was the only contractor that was successful in implementing the Obamacare web sites, and they did it in four states: Connecticut, Kentucky, Rhode Island and Washington State.
I discussed this in detail in the companion article "Healthcare.gov -- The greatest software development disaster in history".
The difference was that Deloitte actually produced project plans and milestones, and then used those to pare down the project to the minimum necessary to succeed on launch day. The other contractors were so incompetent, they couldn't even produce project plans, let alone working web sites.
The other contractors were so drunk with the opportunity to spend hundreds of millions of dollars, they were more focused on spending the money than in getting a working web site. They each hired hundreds of programmers, most of whom were incompetent. They missed one deadline after another, and sometimes lied and cheated on tests.
The other contractors made excuses for why they spent hundreds of millions of dollars but still couldn't complete a simple web site project, but the fact that Deloitte succeeded in four states proves that the other contractors' excuses were garbage.
Researchers would do well to do further study of the differences between Deloitte and the other contractors and derive some lessons about software development projects from the differences.
I believe that it's no coincidence that the "Agile" methodology for managing software development came about at the same time as the rise of Generation-X, since it overcomes some of the contentiousness in Gen-X software development teams. In fact, one web site describes it as follows:
"Born in the early 1990s, agile methodologies share a lot of the same qualities as their Gen-Y peers – bold, youthful, ambitious, flexible, and forever seeking new challenges. This model is fast becoming the project-management method of choice, largely due to its focus on optimizing development time and producing an operational application in the shortest time possible." Clearcode
Agile has its own set of numerous terms like "scrum" and "sprint" that define the details of managing a software project, but in my opinion the most important features of Agile related to this article are:
The new-fangled "agile" methodology is usually contrasted to the old-fashioned "waterfall" methodology. In the waterfall methodology, you start with functional specifications, then development milestones for implementing the features of the specifications. Theoretically, there's no opportunity to change the functional specifications once they've been written, while in the agile methodology, specifications can change at any time, if necessary.
However, there's another aspect of agile that can either work well or cause problems: The daily standup meetings give the development supervisor or manager an opportunity to review progress on a daily basis. If the manager is competent, then he might be able to help by providing resources requested by group members, or get other problems resolved with upper management. In a dysfunctional project, where the manager is incompetent, then the standup give the meetings give the manager the opportunity to micromanage the project and enforce stupid decisions. I've seen both.
There's research that indicates that the agile project management methodology is more successful than waterfall, but it's certainly no guarantee. In fact, apparently most of the contractors on the Healthcare.gov project used the agile methodology, and that was the greatest IT disaster in history. So agile does not protect a dysfunctional project from sabotage or incompetence, but may give a competent manager greater insight into how the development team is doing.
The implementation of the Healthcare.gov web sites was a massive violation of Brooks' Law, with predictable results.
Brooks' Law is well known in the software engineering community due to the ground-breaking 1975 book, “The Mythical Man Month: Essays on Software Engineering” by Frederick P Brooks Jr., the manager of software development for the massive IBM mainframe operating system in the 1960s. (The entire book can be downloaded legally for free in text or PDF form from: Mythical Man Month Download)
When a software development project falls behind, it's natural for management to want to add programmers to it. This is the "ditch digger" archetype of software development -- ten ditch diggers can dig a ditch twice as fast as five ditch digger. Unfortunately, it's usually disastrous.
Certain categories of software development jobs do fall into the "ditch digger" archetype. The most obvious example is data entry. Ten data entry clerks can enter data twice as fast as five data entry clerks, since there's typically no interaction between them; they needn't communicate with one another, and one clerk's work generally doesn't depend on how another clerk finished her work.
Another such category (in most cases) is quality assurance. A collection of bugs can usually be partitioned, and each group can then be assigned to a specific quality assurance engineer. In the ideal case, QA engineers work independently of one another, and don't affect each other's work.
As these examples suggest, the following are the reasons why adding programmers to a late project only makes the project later:
The result of all this is that the skilled programmers end up trying endlessly to train the newer team members, who may not even have the skills to contribute to a project. The result can be disaster.
All of these things happened in the (non-)development of the Mass Health Connector. It was a 5-10 man project, but CGI Corp. was given hundreds of millions of dollars to spend, and did so by hiring 200-300 programmers. As it turned out, these programmers were apparently so unskilled, they couldn't even write specifications, let alone implement anything, and all the code that CGI produced was garbage.
While we're on the subject of software development jobs that can be added despite the restrictions of Brooks' Law, there is a related subject of software development jobs that work best for men and for women. During the 1990s, I did a lot of research on gender issues, and even wrote a book on gender issues for men. So this is a subject that I've researched and is of interest to me.
In order to approach this subject, let's start with the subject of boys' and girls' games. Research by Professor John Mordechai Gottman found that boys and girls had very different playing styles. This is one of those things are completely obvious to anyone with eyes and ears, but are politically incorrect to discuss. But somebody has to discuss them, so I'll do it.
Gottman explored the different playing styles of boys and girls: Boys' games, like cops and robbers, usually involve unrestrained activity or pretend assault, and often contain an element of fear and danger which the boys have to deal with within the game. Emotions can be displayed, but it's never permitted to disrupt the game; the game has to be kept moving.
By contrast, girls' games, like house or hopscotch, involve restrained movements, and seldom introduce fear. The game is not important by itself, but is a context for bringing up, exploring, expressing, and understanding emotions and relationships. Whenever there is conflict that the girls cannot handle they discontinue the game and talk about what happened.
When these observations are extended to online computer games, there are almost as many women as men playing online games, but males are more attracted to the more "violent" first person shooter games. In cooperative multiplayer games, game progress is the only thing that matters to males, and emotions are not permitted to slow down the game. On the other hand, women are more attracted to games that permit sharing and relationships online, often with more time spent sharing than with game play.
Now let's turn to software engineering jobs. I've known some exceptionally good female programmers, so obviously any woman with the core intelligence and the internal motivation can be a good programmer. But software engineering is a very solitary intense thing. A programmer typically spends many hours a day alone at a computer. There will typically be group meetings each day where team members must share very detailed technical with the others. Feelings are not welcome. That's why there are so few female programmers.
I used to teach advanced computer programming to adults at Boston University and Northeastern University, and I often started each class with a little bit of advice: If you're wondering how well you'll do as a computer programmer, then look at how you do your homework assignments. If you sit down in front of the computer or terminal at 6 pm to do your homework, and the next time you look at your watch it's 1 am, then you'll probably do very well as a computer programmer. But if the next time you look at your watch it's 6:02 pm, then you may not do well.
This is to emphasize that computer programming is like first person shooter games, where you have to have to kill aliens and solve problems as an individual, in a solitary activity for hours at a time.
In my decades of experience in software development, there is almost no gender bias in software development groups against female programmers. In fact, there are so few females who WANT to be programmers, most male software developers would be biased in favor of having a female join the group.
Females tend to prefer other kinds of software development jobs besides hard core programming. Jobs that include an element of sharing include documentation, graphics, and qualify assurance -- generally the same kinds of jobs that can be added to a software project without violating Brooks' Law.
I've always been impressed with the technical skills of Indians, ever since I was a math major at MIT and became familiar with some of the work of Srinivasa Ramanujan (1887-1920), who was one of the greatest mathematicians of all time.
However, outsourcing software development to India is not about technical skills. It's about time zones, and culture, and politics. I've seen outsourcing used successfully, and I've seen it used disastrously in a dysfunctional software development project.
Indian software firms might well brag that they "saved the world" in the 1990s, when they did a huge amount of work on the Y2K problem for many companies. A typical situation was that a company had millions of lines of 1970s-vintage Cobol code in IBM mainframe applications that manipulated dates using only two digits for the year rather than four. (That is, the date 1975-09-23 would be stored as 750923.) In order to keep that code running after the year 2000, those millions of lines of code would have to be modified to handle 4-digit years (e.t., 19750923), and the results had to be thoroughly tested.
I wrote numerous articles on Y2K outsourcing to India in the 1990s. In many ways, the Indians were ideal. Little remote training was required, since the code did not have to be entirely understood to be remediated. The time zone differences were actually an advantage, since the Indians would log into the American mainframe during the day in India, but during the night in the US, when the mainframe was usually not busy, so valuable mainframe time was shared effectively. It was the Y2K project that established India as a worldwide IT outsourcing resource, and that continued past the year 2000.
There's been a lot written about problems with Indian programmers, so let's start with the fact that I've worked on projects side by side with Indians and they're just as skillful and competent as American programmers.
But this article, the one you're reading now, is an article about subversion in dysfunctional American projects, so it's appropriate to provide material about issues and subversion with projects outsourced to India.
First, here are some advantages to outsourcing to India:
Most of the difficulties with outsourcing to India occur because all of the problems with violating Brooks' Law ("Adding manpower to a late software project makes it later") are multiplied when the manpower you add to a project is outsourced at all, and are multiplied even many times more when the projects are outsourced to India.
Since this is an article on subterfuge and sabotage in software projects, it's logical to list some of the forms of these that are specific to outsourcing to India. Over the years, the problems that I've observed and been told about the following:
So the bottom line is that outsourcing to India can be good for everyone, provided that be good in some cases, and disastrous in others, requiring close management.
|
As I've said, in my decades as a Senior Software Engineer I've written thousands or perhaps tens of thousands of memos. In most cases these were things like progress reports or requests for resources. In many cases they raised issues, and sometimes even questioned management policies or wanted managers of problems.
Out of these tens of thousands of memos, there were only about half a dozen that got me fired. These are described in the personal anecdotes and observations in this section.
This happened in the 1990s, over 20 years ago. I was working for Fidelity in Boston on a Windows desktop application that would be used by a telemarketer calling investors to help them invest more money. I would be working on part of the user interface and mostly on the mathematical financial algorithms that performed such things as account rebalancing.
When I joined the project, I was told that this was a very high pressure, very visible project, and that hard work and results would count. I welcomed this because I enjoy high pressure projects. There were five programmers, and we were working in a lovely office in the Boston World Trade Center, with large windows overlooking Boston Harbor.
As the project progressed over a period of several months, I complained a number of times to the tech lead, Jeff Doege, that no system testing was being done, and that it was possible that once the different components had been developed, they wouldn't work together.
I wrote a memo to Doege on the subject of "Integration Testing of AAR":
"Up until now, we've only considered unit testing of the modules in Milestone 1. This memo summarizes the additional integration testing that must be performed for Asset Allocation and Rebalancing (AAR). ...As we've already discussed, integration testing is really a new element, and it's hard to predict what will happen. It's possible that everything will work together very quickly, and there will be no major problems. However, I have seen such results occur fairly infrequently in the past, and an estimate of a couple of months for integration testing and debugging might be more consistent with history."
Doege was sympathetic to my concerns, but his boss, Chris Cazer, the software development manager, never showed any interest in end to end testing, nor scheduled it. So Doege suggested that I try to run an informal integration test by myself. So I took a couple of hours to do that, and wrote a memo. I gave the memo to Doege, but since I knew it might be embarrassing, I asked him to be careful whom he showed it to. Excerpts of the memo read as follows:
"Subject: System Testing of AMSI have attempted to perform a system test on AMS, in order to determine how well AAR (Asset Allocation and Rebalancing) interacts with the other modules in the system, in particular the View/Edit Account, Security, Transaction, and Model modules.
This memo documents the result of my attempt.
The short answer is that nothing worked with anything.
I found major bugs and major missing functions throughout the system. In addition, inconsistencies in interpreting data base fields meant that different modules did not work together. ...
Based on my past experience, I reach the following conclusions: (*) A system test of AMS is impossible at this time. (*) There is zero probability that the modules supporting level one functionality will work together within a month. A target date of 3-4 months is more realistic, and six months is not unreasonable. (*) With the current staffing, you should start adjusting your thoughts not to expect this system to be available for production use [until next fall]. With all the functionality yet to be added, I would not expect even a beta test system to be available before summer at the earliest."
This memo also listed dozens of specific problems. Examples of the problems that I listed were: I filled in fields and clicked 'Save' frequently, but started getting dbVista errors (for no reason); Could not enter data into the 'balances' screen; I left the 'Taxpayer ID' field blank, but it was filled in with '-0000000001'; There's no specification for the database fields, so different programmers are using the same fields for different things; There is no error/validation checking on many fields; Inputted data is not checked for consistency with other inputted data; The 'Table Maintenance' option lets you modify all sorts of tables that shouldn't be modifiable, such as the meaning of 'Yes' and 'No'.
This list of problems almost looks like a joke, but in fact, these kinds of errors are common in software development projects when end-to-end testing isn't emphasized from day one.
I also want to emphasize that writing this memo wasn't a huge project. It took me 2-3 hours to examine the source code and try some tests, and then another hour to write the memo. So this was something relatively trivial that a manager should have known.
I look back at this memo, and wonder why I got fired for it. I was doing a lot of development work on the project, and getting a lot of functionality done, and I was fired right in the middle of the work I was doing. A Boomer boss would have thanked me for it.
After I wrote this memo, I gave it to Doege and told him to be be careful where he showed this memo, but of course it got to Gen-X development manager Cazer.
My memory of my meeting with Cazer is a blur after all this time, but the image I have in my mind was that smoke was coming out of Cazer's ears. I realize that's impossible, but that's the image I have after all these years. Cazer was furious, and was screaming at me. This is what I mean when I wrote elsewhere in this article that a Boomer telling a Gen-Xer something about technology can be "explosive." I've seen this screaming in other situations as well.
Another element that's common to these kinds of situations is what might be called "fear of testing." I can only guess that a manager who doesn't approve testing must be afraid that the tests will fail, and doesn't want to know. Perhaps Cazer was ignorant of what was going on, or perhaps he suspected the project was in trouble, but he certainly knew for sure after reading my memo.
This is a situation that goes to the heart of this article, which is about sabotage and fraud in software development projects. I was not a whistleblower; in fact, I was only doing what the tech lead (Doege) told me to do. In similar situations in other projects, especially with Boomer managers, the manager might tell me he was dealing with the problem, and to ignore it for now, and life goes on.
But Cazer's conclusion was not that lack of testing was the problem. Cazer's conclusion was that I was the problem, and he fired me (explosively screaming).
Many development programmers and managers don't understand the importance of integration testing. The incompetents assume that once each component works individually, the components will all work together. In the Mass Health Connector, the managers at UMass and CGI knowingly cheated and lied on the integration tests, according to whistleblower Dave, with the result a total disaster costing hundreds of millions of dollars. In the Maryland Health Connection Obamacare exchange development, the auditor warned the developers of an integration problem, well in advance of the launch date, but the warnings were ignored.
I wrote about these issues at length earlier in this article, in the section entitled "Systems integration issues."
Cazer fired me, and the whole project crashed and burned a month later, and was canceled.
A couple of months after that, I was invited to an informal meeting (in a bar) of programmers and managers who had worked on the project before Cazer took over. At that meeting I learned that there had been powerful groups competing for money, and that Cazer had won the political battle. So it's possible that the reason he was screaming at me is that my memo could be used against him politically. I unintentionally gave his political enemies a weapon, and he'd rather keep the money flowing to himself, rather than end a failing project.
At that meeting (in the bar), I also ran into one of the programmers who had left the project a couple of months before I did. He told me that he had seen that the project was a disaster, and he decided to leave before everyone got screwed. I guess that guy was a lot smarter than I was.
This is another story where system integration issues were ignored, resulting in a development disaster.
In 2006, I was working as a contractor for CACI, which was subcontracting to General Dynamics and the Air Force. I had been working on a large defense project for almost 2 years. The project involved huge amounts of hardware, along with many off-the-shelf software systems that had to be integrated.
It was a strange situation. There were probably 50 engineers working on this project, mostly on various hardware components. Almost none of these engineers knew anything about software, although almost all of them thought they did, and thought that software was trivial. Outside of myself and one other person (Christopher Breeden), nobody knew anything about software or software engineering.
So we had all the ingredients of a disaster. The management assumed that once each of the hardware and software components were working individually, then they would all work together. There was no credible end-to-end testing planned. This situation was similar to the one at Fidelity, only many times bigger.
As usual, I was unable to control myself, and had to point out the problem to my manager. I really dreaded this, because I knew I would probably be fired, since there was too much money involved and managers didn't want to hear about problem, but I had to do it anyway because of some weird personal compulsion. Here are some excerpts from the memo I wrote to my CACI manager, Steve Rogers:
"I've mentioned [to you in recent weeks] that the project is running into problems, but now I'd like to describe the situation explicitly.The project has 0% probability of being ready for FSE; it has less than 5% probability of meeting FSE requirements by January; it has about 50% probability of working at all, even with an extended schedule.
This assessment has been reached by a method which always produces correct results: As someone who sits on the "bottom" of the management/supervisory tree, I can see many things that are invisible to upper management, even first level managers. I can assess the project's progress since I've been here, and I can extrapolate that progress forward to reach a probability of success. ...
As the Senior person on this project (and possibly the only person) familiar with the implementation of large software systems, I've frequently been surprised by becoming aware of significant integration problems over a number of months. Today, none of these problems has been solved, nor appears to have any chance of being solved soon.
Last week I conducted an informal survey of other members of the group. Here are some conclusions -- and these things are only the tip of the iceberg:
(*) We now have a collection of "point" software products, each of which may (or may not) work on its own, but is completely unintegrated with any other products. (*) This situation has been exacerbated by the overnight emom switch [which] presented a new set of major integration issues that ...had not previously existed. (*) I'll start with my own area of responsibility, since I'm most familiar with it. The Consolidated Web Server (CWBS) server has been properly installed, as far as I know, but the exact configuration has not been finalized, and it's installed on an incorrectly configured server. ... The installation process was almost completely undocumented or incorrectly documented, and the installation software was buggy, sloppily written and complex. I documented a dozens bugs and problems and sent them off to the Netcool people, and received only the response that I should take a course. ... (*) The IBM Netcool emom appears to be installed, but there are components missing, and Netcool is not integrated with other products. Netcool appears to be collecting events from some devices (which is what the FSM is supposed to be doing), but it's not receiving and integrating events from other sources (which is what it's supposed to be doing). The Netcool component list appears not to have been finalized. (*) I'm told that the Netiq HSM is installed and working, but right now it doesn't "manage" a single host system, and it's not clear that it will even work with some of them. Also, it's not forwarding events to the Emom, as far as I know. (*) Remedy appears to be the only component that's integrated with anything else. It took Breeden a few months to get it working, but Remedy can now open trouble tickets for some events that it receives from the Emom. However, Remedy is not correctly integrated with the CWBS. (*) Another component that I'm responsible for -- QC testing the SQL Server installation -- has been stalled for several weeks because the installation servers are down and have been down the entire time. There have been daily promises that they'd be working "today or tomorrow," but nothing ever happens. (*) The Veritas Backup component is installed, but is not backing up, let alone recovering, anything. If there were nothing else to do, this alone would require several weeks of configuration work, since the backup component requires close integration with other components, and the integration design has never been done, let alone implemented. (*) The Failover function is a huge job which has not even been attempted. If there were nothing else to do, this would require 2-3 months, since it exposes bugs and problems in the entire design.
I call your attention to the last two items (backup and failover) particularly: They aren't even designed, let alone implemented; we can't begin implementation until the previous problems are resolved; and whenever we begin implementation, it will take several months to complete.
Thus, the earliest that this implementation can be completed is roughly Q2 of next year, and Q3 is more likely. No earlier completion is possible. ...
You're already well aware that the documentation is behind schedule, but you should be aware that there are enormous chunks of content missing, and they can't be completed until the design is finalized, so the documentation won't be ready for FSE until Q1 at the earliest. ...
There is one thing that I can offer, if you're interested and willing to support it within CACI and GD. I could do a thorough project review, talking to everyone (mostly engineers), producing a complete report of what's been done, what has to be done, how long it will take, and the probability of success. I can do this within about two weeks, provided that I have your support and GD's support to do it. However, I'm sure you're aware that I'm a purely technical person, with almost no political skills, so if I do this you're well aware that I'd break a few eggs. The advantage though -- and it's a big advantage -- is that at the end of the two weeks you'll have a complete and accurate picture of the project status, something you don't remotely have now. So even with all the political disadvantages, things are now so bad that this is something that you might be willing to consider."
The last paragraph is laughable, as if those Gen-X managers in General Dynamics would ever stoop to allowing a mere Boomer do such an analysis.
I believe that I could have saved this project, and saved everyone's job, and saved everyone $10-15 million, if they had let me do my job and do this evaluation, and if they then followed my recommendations. But that's not the way things work in the nihilistic Gen-X world.
This memo that I wrote was almost laughingly deferential, since I was afraid of being fired. In fact, my manager Rogers thanked me for the information. I breathed a sigh of relief, thinking that I wouldn't be fired.
But a week later, a General Dynamics manager, whose name I can't recall, started screaming at me. All I recall is his screaming that "THIS PROJECT WILL BE COMPLETED IN FOUR MONTHS." And I was fired a week later.
As I've said earlier in this article, screaming seems to be something of a constant in dysfunctional projects. I was screamed at at Fidelity, at CACI, and at Ability Networks as I'll describe later. And it was just for writing a memo that pointed out problems in a project.
Needless to say I was right, as I almost always am in these situations, and the manager who had screamed at me was wrong, and the project didn't finish in four months. It went on for 18 more months, and finally completely crashed and burned. It was a project worth over $10-15 million dollars, and nothing was salvaged. It was a financial disaster all around. If they had listened to me, I might have saved the project, but that's not the way things work.
This was a much bigger and more complex project than Healthcare.gov, but at least it cost "only" $15 million, not $5 billion.
But it shows all the usual common elements of these disasters: incompetent technical management, no understanding of software integration issues, no understanding of the need for end-to-end testing, considering it to be a crime to point out that a project is in trouble, and then a dénouement of someone screaming at me at the top of his lungs. I wrote about system integration issues at length earlier in this article, in the section entitled "Systems integration issues."
So, once again, Dear Reader, if you're in the position where you're working on a failing dysfunctional project for an incompetent or unethical or crooked or corrupt manager, you probably should be smarter than I am and just quit, unless you're willing to cross over to the dark side and just join in the lying and cheating.
I'll tell this story because it's instructive, about an interview.
In 2008, I interviewed for a contract programming position at OneShield, which provides software products for insurance companies. I did very well on the phone screen, and was invited in for an interview. I was being interviewed by the manager John Rikala, and thought it was going well. Then another manager came in the room, Ruth Blaney. Blaney questioned me closely about past experience, and along the way, I described my experience at Fidelity (described above).
She said, "You told your manager that the project was going to fail?" I said, "Yes, as a professional consultant, I felt it was my duty to do so."
At that, she looked at Rikala, said something about coffee, and left the room. Rikala terminated the interview, and I was sent home.
I was extremely depressed by this interview because I didn't have the vaguest clue what had happened. I was so depressed that I sent an e-mail message to Blaney and Rikala:
"I'm writing to you to inquire about your reaction to the story that I told about my experience at Fidelity in the 90s. I was responding to a question you asked. As you may recall, I notified my boss at Fidelity in a memo that the project was in serious trouble and that it would slip 3-6 months. He was furious at me that I told him this (especially because I was right). He fired me a few days later, and the entire project crashed and burned two months later.I told you this story to illustrate that I'm a good team player and that I keep my management informed about what they need to know, so that they can take corrective measures as soon as possible, reducing project risk. I also wanted to illustrate a sense of personal and professional ethics.
Your very strongly negative and critical reaction to my story has completely confused me.
I've already asked several people why you might have reacted that way. One suggested that you thought I was being disloyal at Fidelity, which doesn't make sense to me. ... And another suggested that perhaps you thought I should have worked on the problems with the other team members, without even telling my manager.
None of those answers makes sense to me, and I really don't understand the reason for your strongly negative and critical reaction, and I'm wondering if I can impose on you to help me understand why."
So I sent that e-mail message off, but of course I received no response. You can draw your own conclusions, Dear Reader, as I have. And people wonder why software projects fail.
This is a different kind of story. One big difference is that I didn't get fired, even though I wrote some memos. But the memos were ignored for a long time.
Two programmers were sabotaging the software by deleting code. If they hadn't finally been found out, they would have put the public into danger. They started by deleting my code, but only got into trouble much later when they deleted someone else's code.
I worked for three years for CSC, which was under contract to the FAA. The particular project was Traffic Situation Display (TSD), a Unix-based C++ system used in control towers across the country to track passenger airliners in real time. The UI displays a map of the U.S., and each dot on the map represents an airliner's present position. You can zoom in and get detailed information about any flight in real time. The map also displays numerous weather conditions, such as wind, rain, snow and lightning.
This was the most bizarre situation I'd ever seen. All the functionality that I added to TSD contained data validation code, which is critical in an application like this one where public safety is at stake. The two project leads, who disliked me personally, deleted all my validation code, apparently believing that validation code was a "Boomer thing." But they were putting the public in danger. I had never seen anything like this and was totally flabbergasted that they were doing this, and I was even more astonished when they were going to get away with it.
This went on for months. I wrote numerous memos to my manager Art Karpinski, but they were ignored since it was their word against mine, and he didn't believe me.
After three years, the project was being moved to New Jersey, so I left for another job. Before leaving, I wrote a memo to my manager:
"As I started in TSD development, it became clear to me that no validation checking was being done by TSD, and that there were many places where bad data would crash TSD. Also, this problem has gotten significantly worse in the last three years, as new data feeds were being added. So in all the new functionality that I developed, and in all the many bug fixes that I implemented, I always put in plenty of data validity checking.This code greatly infuriated [the two programmers], who began complaining that I wasn't "adapting" to their style of programming, which was to ignore error and validity checking, and just let TSD crash. ...
They particularly targeted all the error and validity checking code that I had implemented, and removed all of it over time.
I've been a software engineer for decades. I've worked for dozens of clients and employers, and I've helped bring hundreds of software projects to completion. I've worked alone, in small groups, in large groups, as project leader, as project follower. You name it, and I've done it.
But I have NEVER seen anything like this -- programmers wilfully deleting another programmer's code out of revenge, especially validity checking code. I was absolutely flabbergasted that this could possibly go on, and I became even more astonished as it became clear that they were going to get away with it.
That's why I wrote numerous memos to you describing what was going on, documenting new bugs and problems that they were causing. I wanted everything to be "on the record." Furthermore, I documented numerous lies that [they] were telling on a daily basis, as well as the massive amounts of information that they were withholding from you. I wrote that if they had tried this at many of the other companies I'd worked at, especially the financial institutions, then there would be firings and possibly arrests. I won't recount the expressions of disbelief that I heard, but I am grateful that I wasn't fired, though I'm still surprised that I wasn't.
However, the result for TSD is that a lot of code is a pile of crap. ... The code is unstructured, violating TSD's ... own programming standards. The code is poorly commented, with highly interlocked spaghetti code. The class structure is haphazard and non-intuitive, violating accepted standards for separating computation and GUI code.
TSD's code is like a house where the electrician had strung wires every which way, with no logic or fuse boxes. All the light switches work, but if you plug in an extra appliance, then the house burns down. And every time that I added a "fuse box," they deleted it.
I don't believe that you grasp the consequences of this situation, so let me give a specific example.
A couple of weeks ago, I brought up TSD and it crashed immediately. There was no obvious workaround to the problem. ...
I entered a debugging session, and determined that the crash was caused by bad lightning data being fed to it. I understand that some of the testers experienced the same problem, and it was blamed on bad lightning data.
The problem is that the TSD code contains no validation checking for the lightning data at all, and bad data causes TSD to crash.
Now suppose that TSD is installed in centers across the country, and someone sends out bad lightning data, or bad data in any of the other numerous data feeds. The bad data might be caused by a programming error, or it might be a perfectly legitimately format change that wasn't properly implemented.
The result would be that ALL TSDs across the country would crash at the same time, possibly causing a crisis that would put lives in danger.
And this is not a far-fetched scenario. It's already happened here, and I've gotten other TSD crashes from bad data. It could happen at any time in the field -- in a couple of months, or next year, or the year after that.
Furthermore, the situation is made MUCH WORSE by the fact that TSD is now moving to CSC N.J. where, as I understand it, no one will have the vaguest clue how TSD works. Thus, if a crisis does occur, then CSC NJ will look like complete idiots. ...
I can assure of this: If I had been able to do the job that I was hired to do, then TSD would be a very solid product. Every data feed and every data interface would have a rock-solid implementation, with complete data validation, and the ability to withstand anything thrown at it.
Instead, we have a shoddy, fragile product that cannot withstand stress, and in some scenarios may be a danger to the public. As a professional Software Engineer with decades of experience, I cannot find the words to express how sickened and revolted I am at this situation. It almost makes me want to vomit.
That's why I wrote 18 months ago that if this had happened at other companies, then there would be firings and possible arrests. This is a very serious crime."
After I wrote this memo, the New Jersey group complained that the same two programmers were deleting THEIR code as well. Finally, Karpinski apologized to me.
I don't blame Karpinski, since he was duped by the two programmers, and didn't fire me. But that makes the situation all the more serious, because it shows how criminals can use technology to deceive their management, even when someone else on the team repeatedly points out what was going on.
For more information about this kind of subversion, see the earlier section on Generation-X attitudes towards Boomers and technology.
For academic researchers, I think that the most remarkable lesson to be drawn from this experience is the resilience of the dysfunction. The situation I've described went on for almost two years, and I wrote dozens of memos, but the situation continued unabated. For me personally it was really depressing because I was expecting to be fired almost on a daily basis. One Friday I even called in sick because I thought I would be fired that day, and I thought I'd force them to wait until Monday to fire me. I didn't get fired, but the situation continued month after month after month. How is that even possible? Some researcher better than I am will have to answer that question, because I don't have a clue.
I'm going into greater detail in the Ability Networks story because so many issues were raised, and each issue gives me a chance to expand on earlier topics. This story should be a gold mine for academic researchers, because I think that every management mistake in the book was made.
The research topics related to this story involve unit testing, system testing, outsourcing to India, and doubling down on bad management decisions.
I had been working at Ability Networks for almost a year on a project called Ease, a Medicare claim submission system sold as Software as a Service (Saas), with thousands of customers. I really loved Ease because of its importance and complexity, but then a new software development manager Geoff Charron, and a new Ease development manager, Andy Kmett, took over, and things changed for the worse overnight.
Charron's first decision was to cancel all use of the "Medicare Simulator," which had been a part of Ease since the beginning, and was the only testing tool available. I was flabbergasted. It quickly became clear that Charron and Kmett knew absolutely nothing about software engineering, and certainly absolutely nothing about Ease. I'd been through enough experiences to know that there was a good chance that sooner or later I would be fired, since I work very well for intelligent, competent managers, and very poorly for idiots.
Before Charron appeared, I had been working on a path with my previous manager (Jeff Forsyth) where I would add some enhancements to the Medicare Simulator, and then work with the team to use it to help in developing and testing a major enhancement to Ease itself, the "large hospital problem." Adding this enhancement would have meant millions of dollars in additional revenue to Ability. But with any use of the Medicare Simulator forbidden by Charron, all of this was out the window, and Ability lost millions of dollars.
One theme that runs through dysfunctional projects that end in disaster is that the management has no idea of the need for testing, especially end-to-end systems integration testing. I wrote about this at length in the "Systems integration issues" section, and we saw this in the Fidelity story and the CACI/General Dynamics story , as well as the Maryland Health Connection Obamacare exchange disaster. In all of these cases, management thought that if each component worked, then all components would automatically work together. We also saw it in the Massachusetts Health Connector case, where testing was replaced by lying and cheating and firing the person who refused to sign off on the lying and cheating.
Managers who know little about software engineering do not grasp the difference between "unit testing" and "system testing." Someone like Charron, whose background according to his resume was in mechanical engineering but who had dabbled in a little programming, undoubtedly had experience with unit testing, which each programmer does on each section of code that he writes. But if a manager has never worked on a large software system, then that manager will have no grasp of how complex systems fail. How Complex Systems Fail, Richard I. Cook, MD, Univ Chicago (PDF)
With Charron's decision, system testing was essentially thrown out the window, and amazingly, changes had to be tested directly on a live production system. Over the next few months, the only way to test back end changes to Ease was to put them into the live production system and see if they worked. Once again, as in the case of these personal anecdotes, I was totally flabbergasted. To test fixes on a live production system with thousands of users when there was a perfectly good testing tool available was absolutely insane to me.
The way that Ease works is to pretend to be an IBM 3270 terminal (1960s technology) and use screen-scraping technology to log into the IBM mainframe system in Washington that handles Medicare claims. In order to test on the live production system, it was necessary for the development group to go to an Ability customer and borrow from him his login credentials to the Medicare mainframe system in Washington, and use those for testing, since the Medicare people didn't permit test accounts. If the changes to the production system didn't work, or if customers complained, then the changes had to be backed out which, fortunately, could be done quickly.
Incidentally, as I'll explain further below, I continued using the Simulator for testing, and so I was able to test my bug fixes and new features on my local computer without having to use the live production system. Charron apparently was unaware of this, but of course the other group members were, and the tech lead assigned me several bug fixes and features that only I could implement because only I could test the fixes.
In the midst of this increasingly chaotic environment, Charron had a financial relationship with an Indian outsourcing firm, and he wanted to hire as many Indian employees as possible. During a morning standup group meeting, Kmett said that they would hire seven more programmers, mostly from India. By the way, these morning meetings were conducted by phone, since the main team was in Minneapolis, Kmett was in Connecticut, and I was in Boston, usually working at my apartment in Cambridge.
I told Kmett that this was a very bad idea, because Ease was one of the two or three most sophisticated and complex software systems I've ever worked on, and adding seven programmers could only lead to disaster. I referenced Brooks' Law. But I might as well have been talking to a brick wall.
For academic researchers who are studying outsourcing to India, this is a prime example worth studying. I wrote about this issue at length in an earlier section of this article entitled "Outsourcing software development to India."
So Charron and Kmett brought a group of Hyderabad programmers on board, and neither Charron, Kmett or the Indians had any clue at all what they were doing. Every problem that I identified earlier in this essay with violating Brooks' Law or with hiring Indian programmers occurred to Ease. It was a complete mess:
One more thing: Since the Simulator allowed system testing of Ease, it also served as a training tool for understanding how Ease worked. If the Simulator had been available to the Hyderabad team, then they could have trained themselves much more, and the Minneapolis team would have been impacted. This article is on how "subversive stakeholders" in a software development project can sabotage a dysfunctional project. It's amazing how thoroughly Charron served as a subversive stakeholder and sabotaged his own project.
After a couple of months, the Indian team provided a collection of nine bug fix patches to the Minneapolis team. It was immediately clear to the Minneapolis team that these fixes had been provided without understanding how the system worked. Four of them were clearly wrong, and were rejected immediately. The other five were tested by moving them into the live production system, but created immediate problems, and so all of them had to be backed out of production. The whole thing was a total disaster.
During a standup meeting, Kmett said "I've certainly learned a lesson." However, it was never clear to me what lesson was learned, since nothing changed. I suggested again that the Simulator could make the Indian team a lot more successful, and I suggested that it be revived. But once again I might as well have been talking to a brick wall.
I've observed differences in Boomers and Gen-Xers in the way they react when one of their previous decisions is proven to be a bad decision. What I've seen among Gen-Xers is that once they make a bad decision and it's proven to be a bad decision, they double down on the bad decision rather than changing it. This is the opposite of Boomers, who are much more likely to change a bad decision when it's shown not to work, in my observation.
Boomers grew up in the 1960s and 1970s, when the youth culture was to criticize the government. As adults, Boomers are frequently poor decisions makers, because they're easily dissuaded from any decisions they make. Gen-Xers are the opposite. Many Gen-Xers are children of divorce, having grown up with no fathers, except for a string of men in their mothers' beds, some of whom were undoubtedly abusive. As I wrote earlier in "Generation-X attitudes towards Boomers and technology," Gen-Xers have a visceral hatred of Boomers, as can be verified by anyone spending half an hour on the internet. This apparently translates into a specific kind of decision-making behavior: A Gen-Xer asks himself what decision a Boomer would make, and then he makes the opposite decision. Since Boomers are right more than 50% of the time, this means that Gen-Xers who use this process will be wrong more than 50% of the time.
As I described earlier in the section "Generation-X in job interviews: 'Dumb down your resume'," if you want to get a job, then to stay on the job, the Boomer employee has to pretend to be as dumb as his Gen-X boss, or his Gen-X boss will fire him, simply because he's older. I call this "glorifying stupidity," and it's not only soul-destroying advice, but it's incredibly destructive. Charron's decision to cancel the Simulator, and then to double down on that decision, was due to the fact that it was being promoted by a Boomer (me). Being a Boomer was undoubtedly also the reason that I was fired, as explained below.
Since there's a tendency among Gen-Xers to ignore how destructive they've been, let's just review the results of Charron's decisions so far:
Just to be clear, this was caused by a series of very bad decisions, starting with the decision to cancel the Simulator. Once the Simulator decision was made, the other decisions were examples of "doubling down" on the original decision. If Charron had reversed himself, and the Simulator had been available, then the "large hospital problem" could have solved, bringing in millions of revenue; the Hyderabad team would probably have been successful, since they would have had the Simulator as a training and testing tool; and the Minneapolis team would have been much more productive.
After that, the Indian team was assigned only GUI fixes, since those could be easily tested without the Simulator and without customer login credentials. Even so, every fix had to be carefully reviewed by the Minneapolis team, which meant that each fix took twice as long as necessary.
What was clear to me was that Charron was just stumbling and lurching from one bad decision to the next, though, incredibly, he believed that he knew more than anyone. The project had turned into a clown circus, and I was becoming increasingly cynical and jaded. The lead tech on the project, with whom I worked closely, assigned to me most of the things that had to be fixed in the back end, since I was the only one who could work on them, since I was using the Medicare Simulator, and so I was the only one who could test the fixes before they were moved onto a live production system.
As time went on, even mentioning the Simulator was a touchy subject with Kmett, but since it was part of my job I had to mention it in some of the standup conference calls, and when I did, Kmett would pretend that he didn't hear me. It was unbelievably pathetic and sickening, and almost made me want to vomit.
Since I'm not particularly good at hiding my feelings, Charron and Kmett soon decided that they had to take action. At one point, Kmett sent me a warning memo that I shouldn't criticize management. I wrote this in response:
"Ease is an aging product with serious design flaws being pushed farther and farther beyond its original intent, and Charron treats it like a college homework assignment. Pretending that there's no problem, or getting rid of the messenger (me), if that's what you're considering, would not solve your problem. There's already a fair amount of discontent about management within the development team. Since I'm a particularly bitchy person, I sometimes informally take on the role of saying things that others are thinking, but are uncomfortable saying. Ordering people not to complain about management is self-defeating and if enforced will motivate people to leave. This isn't Iran. You have to confront problems, not bury them or pretend they don't exist.I hope you had a nice weekend."
I was actually being very careful in this paragraph, since I didn't want to say anything that would embarrass the other team members. But it was not me who was telling the other team members what to think. It was the other team members complaining to each other and to me about the farcical situation. Something I had done frequently in my career was to take it upon myself to be the one to tell management things that other people were saying to each other but not to management. I considered this part of my job as the Senior person on the project. Obviously, I knew that I might be fired for doing so, but I also knew it was my job, and I would be fired sooner or later anyway. But there was always the hope that Charron might actually pay attention rather than doubling down on bad decisions, but that never happened.
I always consider myself a team player, and the group would have been much better off if Kmett had actually listened to me. If Charron and Kmett had let me do the job I had been hired to do, and if they had taken my advice seriously, the group would have been much more productive, and Ability still might have made that millions of dollars in extra sales. Instead, they decided that I was the problem, and that if any of the team members were unhappy, it was because of me.
Since Charron and Kmett couldn't admit that they were sabotaging the project, they had to find blame elsewhere, and of course that was me.
One thing that puzzled me was that they had no idea about group dynamics. Didn't they have to take some management courses before becoming a manager? These people were not only incompetent as technical managers but also incompetent as managers. They appeared to me to be as dumb as a bag of hammers, and what's really laughable is that, incredibly enough, they actually thought that Millennials and Gen-Xers on the development team would listen to me, a Boomer, if I told them to be unhappy about something. Anyone who believed that I could convince a Gen-Xer or Millennial of anything like that would have to be completely out of touch with reality, but they apparently did believe it, and decided that I was the cause of all their problems.
Other things were going wrong too, and Charron and Kmett always had to find a patsy to accuse. One of the biggest problems was that since testing could only be done on the live production system, it meant that the production system had problems when fixes had to be backed out. That didn't happen with my fixes, since I tested them with the Simulator, but it did happen to other people's fixes. And when the system had a problem, Charron had to find someone to blame.
He accused one person, a very talented Indian programmer working in Minneapolis, of causing the production system to be down for a while. As it turned out, the accusation was completely bogus, but that programmer got so pissed off that he quit and left the company. In addition, Charron also got rid of a valuable analyst, as well as another Indian programmer, giving as a reason that he wanted to get rid of contractors. Apparently Charron wanted to give the jobs to his pals in Hyderabad, who were also contractors.
I was actually very productive at this time, fixing bugs and adding new features to Ease. That's because I was using the Medicare Simulator to test things. I could go through five or ten testing iterations in an hour, changing code and then seeing how it worked, while the other team members, who could only test on the live production hardware, could only go through one or two test iterations a day. So obviously I was a lot more productive than anyone else. Quite honestly, I couldn't see how the Minneapolis team could get anything accomplished at all. But the whole management situation was so farcical that I was becoming even more jaded and cynical.
I'm embarrassed to include this next memo that I wrote because it was such a stupid thing to write, but it was late at night, and I was sleepy, and it came after I and the other group members received a new memo that imposed additional restrictions on using one additional server, the "Adjunct server" for testing, making testing even more impossible:
"OK, gang, no more using the Adjunct server for debugging. No problem. Just use the Medicare Simulator for debugging.Oh wait, the Medicare Simulator project was canceled, and we were told that the Simulator is a waste of time, because we can just use the Adjunct server for debugging.
So what do we do now?
I suggest that everyone go out and buy an Abacus. I read somewhere that the Chinese have developed a new multi-processing abacus with extra rows of beads. You can load Ease Server into one of the rows of beads, the data base into another row, and Eclipse into a third, and you're all set to go."
Well, I thought this was very funny, and so did the other team members, but apparently Charron was absolutely furious.
During this period, I was the most productive person on the team, since I was using the Medicare Simulator and could quickly implement and test bug fixes and new features. The other team members were hobbled by Charron's bizarre decisions about servers, while the people in Hyderabad were restricted to testing and straightforward UI fixes. Still, productivity is irrelevant in a dysfunctional software development group, and nothing could save me now.
The dénouement was quite dramatic, and worth retelling.
I received a warning letter from Kmett saying:
"The purpose of this memo is to address your ongoing communication issues, specifically, sending email messages with inappropriate and negative commentary to team members and others outside the Engineering team. ... It is important that you recognize and understand the severity of this situation, as ABILITY will not tolerate these behaviors from you moving forward."
The warning memo included some quotes from my memos, including the "abacus" memo above, which apparently was the thing that really infuriated Charron. This warning memo looked like a complete joke, and would be an embarrassment to any company that wasn't completely dysfunctional. But these people were serious.
Kmett called me to discuss this, and he added onto the phone call Jessica Halverson from Human Relations. It's amazing to me that with one disaster after another happening with the group, they decided that the problem was that I made a joke about an abacus.
So Halverson was explaining to me what an evil person I was. I knew I was close to being fired, but I didn't want to be fired, so I decided that I would apologize and promise never to do it again. I figured this would buy me a few weeks.
So I started to say the following sentence: "OK, I'll do whatever you want, I'll tell the other members what happened, and then I'll never talk about it again."
Halfway through that sentence, Halverson started screaming at me. "Why are going to talk to the other members?" Not for the first time, I was totally perplexed. I said, "Well, they're both my friends, and I work with them all day long, and I'm an open person and share what's going on."
I don't recall much of the rest of it, except that Halverson continued screaming at me, almost hysterically. I don't recall that I've ever met Halverson in the past, but any man reading this will know that when a woman you don't know keeps screaming at you, then there's something else going on with her that has nothing to do with you. It was obvious to me that Halverson hated older men, and that my age had a lot to do with what was going on, including being fired, and that Halverson would not have treated a younger man the same way. So I was in a completely ridiculous situation, with a hysterical woman screaming at me, telling me that I shouldn't talk to my two co-workers, as if that were even possible, given that I worked with them 8-10 hours per day. Telling me not to talk to my co-workers was delusional and irrational. As usual, Kmett was totally worthless, saying nothing and just let the Halverson farce unfold.
So I said that I couldn't promise not to talk to the other team members. Obviously. The next day I got another call from Kmett and Halverson. Halverson started screaming at me again, and after a couple of sentences it was clear they were going to fire me. I decided I didn't want to listen to any more of her crap, and I certainly didn't want to listen to her screaming anymore, so I hung up. After that, we conducted the rest of it through e-mail. I was actually very sad about all this, since I had really enjoyed working on Ease, but these people were a mess, and I didn't have a chance.
I've gone into a lot of detail in this story in order to have an opportunity to expand on some previous thoughts.
First, there's this issue of being screamed at, which happened at Fidelity and CACI and now at Ability Networks. In all three cases it was a shock to me because I was just doing my job, albeit the third case was so incredibly awful and farcical that I was much more jaded and cynical than in the previous cases.
What is it about writing a memo describing a problem that causes these people to lose control and start screaming hysterically? The only explanation I have is to get into pop psychology and conclude that it's a subconscious admission of guilt. Perhaps readers have some thoughts on this. At any rate, see the previous section on "Dysfunctional projects: Paranoia, abuse, firing, screaming."
Returning to the wider issue of whistleblowing, as I've said repeatedly I've never considered myself a whistleblower when I wrote a memo to my management describing a serious problem with the project under development. In fact, I've always considered it part of my job, and as I've said, I've probably written many thousands of such memos over the years. In the last few sections of this article, I've been documenting the very few that have gotten me fired.
This is the point: The memo-writer isn't a whistleblower unless the manager turns him into a whistleblower, by treating him abusively or by firing him. And, in my observation and experience, the only reason that a manager would treat abuse or fire a memo-writer is if the memo-writer was right.
So my advice to CEOs and others who are responsible for big software engineering projects is to watch for a major red flag: If a manager is obsessively secret, and treats employees abusively, then your software development project may well be headed for disaster. I've also seen many software disasters in my career, either because I participated in them, or I observed them, or I reported on them (as a tech journalist), and this abusive secrecy is a common theme in all of them. If a manager can't stand to hear the truth, then he's almost certainly hiding something important, and probably not competent to be a manager.
Update:
As I was finishing up this article, I just heard from Jim Newhouse, one of my co-workers on the Ease team, a Boomer and an extremely skilled software engineer. He'd been on the Ease project for years, and was the only remaining expert on the intricate functioning of the highly complex back end. He's just been fired for no discernible reason except for his age. Charron finally got rid of all the Boomers, once and for all. Jim says that a new version of Ease was just about to be moved into production. Jim always had the responsibility to monitor and test new versions, so it's possible that Jim was fired at this time to prevent him from performing those tests.
There are some interesting statistics that come out of this.
The Ross/Glass book indicates that "stakeholder subversion" occurs in 20% of all software development projects. In my personal experience, having worked on hundreds of projects for dozens of employers, about 4-5% of software development projects are totally dysfunctional. The two figures are consistent, since stakeholder subversion does not necessarily mean that a project is completely dysfunctional.
In fact, it may be that the difference between a "normal" project and a dysfunctional project is that the normal project can handle stakeholder subversion, while a dysfunctional project cannot, especially when the stakeholder is the manager.
My main conclusion is that once a software development project becomes completely dysfunctional, then it's pretty hopeless. The incompetent manager will use subterfuge, deceit, self-deception, abuse, firing, and any other method to keep the dysfunctional effort going. This is pretty obvious when you consider the Bernie Madoff situation, where whistleblowers were ignored and Madoff defrauded people of $60 billion right under the public's noses.
First, let me repeat: I've worked on hundreds of software development projects, and out of those hundreds, there were only five that were dysfunctional, the ones that I've described. In addition, the Obamacare exchange projects that I described in the companion article were almost completely dysfunctional. But the vast, vast majority of software development projects are NOT dysfunctional.
When a software development group is dysfunctional, then dysfunction is too powerful to overcome. It's a mental health problem, involving a combination of delusion, self-delusion and paranoia. This is particularly true when combined with toxic generational attitudes, particularly from Gen-Xers.
The stories that I've told in this article and the companion article are so incredible that they're almost beyond belief. You have managers cheating on tests and lying about them, lying about enrollment, ignoring code deletions by programmers, ignoring schedule deadline misses that mathematically can't be made up, and being completely ignorant about testing and testing tools.
This stuff is all Software Engineering 1.01. Someone who claims to be a software engineering manager should at least know these basics, but the ones I've described didn't. To excuse these managers is like excusing a building architect for not realizing that a house has to have wiring and plumbing. These are all competency issues, and a software development manager who doesn't understand them is not only incompetent but is also delusional.
As far as I can tell, many Gen-Xers consider system testing and validation to be a "Boomer thing," something to be loathed and avoided. You have to be incredibly stupid and delusional if you're a software developer or software development manager, and you think that system testing and validation are "Boomer things."
Let's take a look at a list of "Five lessons learned from HealthCare.gov" from Federal Computer Week:
This is the typical list of bromides that you would find in any popular newsletter, particularly one that didn't want to risk insulting the Obama administration and suffer retribution. Federal Computer Week
Those are all pretty good rules, but they're useless for the dysfunctional projects and delusional managers we're talking about. In the case of Healthcare.gov, you can't assign just one leader in a $5 billion project; you can't address the whole experience on a $5 billion budget; you can't structure budgets in a $5 billion project; and if you have $5 billion, then you have to hire thousands of programmers, most of whom will be too incompetent to even understand agile, iterative practices of a flexible hosting environment. The problem with Healthcare.gov was too much money, which led to criminal fraud, which the Federal Computer Week article doesn't even address.
So as I said, there's no solution. It's like trying to reason with a madman. If you believe that the world is out to get you, then reason doesn't apply. Or it's like trying to reason with a habitual drug addict. Or it's like trying to reason with a habitual crook. Or it's like trying to reason with a mother who beats her children. None of these situations gives way to reason, nor do dysfunctional projects.
I've seen too many things that I can't unsee, too many things that are dysfunctional beyond the mind of any rational human being. These are documented in this article and the companion article.
Here's another one. During the 1980s, I was working at Northrop, and I mentioned to a program manager that an embedded system project was going to slip several months. I wasn't fired, but I wasn't believed either. To me, this was glaringly obvious. How many milestone dates does a project have to miss before you can be sure the release date will slip? This project was missing them all. Even a week before the release date, the lead programmer claimed the project was on schedule, and even gave a demo. So a week passed, and the project wasn't ready. It ended up slipping six months. I was kept around because it turned out that I had been right.
For decades since that time, that has been one of the strangest events of my life. Was the lead programmer lying, or did he really believe he could make up months of work in one week? If he lied, what was his motive? Why couldn't the other managers draw the right conclusions from missed milestones? It was my first direct experience with a truly dysfunctional group and delusional people, and it's been a mystery to me my whole life.
After that experience, and many others, including those I've documented here, I give up. There is no solution. Dysfunction is too powerful and too resilient to be overcome.
Disasters are going to occur. If you're a CEO, you may think you don't want disaster to occur, but I disagree. You'd rather have the dysfunction. It's in your comfort zone. Enjoy the dysfunction now, and hope disaster won't come, or if does come, it won't come for months or years, after you've gotten your bonus, and you can deal with it at the time. When that happens, you can just get another job, enjoy another few years of dysfunction, and repeat the process when that leads to disaster.
That's what's happening in the banking industry. The bankers who knowingly created tens of trillions dollars in fraudulent sub-prime backed securities kept the money they made through the fraud, and were never punished. They didn't care about the fact that millions of families went bankrupt and lost their homes. "Who cares about them when I can make millions of dollars through fraud and have sex with as many women as I want?" So they then went on to commit other kinds of fraud, as has been proven by the Libor and Forex frauds. The same kind of thing happens in the computer industry. An incompetent manager can go on creating one dysfunctional software project after another, and keep going as long as he gets paid.
What about having an oversight group monitor the development group? Will that work? Well, Bernard Madoff had plenty of oversight, but nobody believed the overseers, and Madoff made $60 billion in his Ponzi scheme swindle.
Another example was the Mass Health Connector. CGI Corp. was the developer, and University of Massachusetts Medical School was hired to do oversight. The whistleblower Dave worked for UMass as part of the oversight effort, and he was fired for refusing to sign off on a fraudulent test by CGI. So someone else at UMass signed off on the fraudulent test, and the oversight group was just as bad as the developer. So having an oversight group just means that the oversight group becomes part of the dysfunctional effort. It accomplishes nothing except to cost more money.
I suppose if you're a CEO and you really want to try something, you might ask a skilled personal friend to come in evaluate a software project independently. It has to be someone you really trust. But I'm pessimistic about even that since everyone can get sucked in by the dysfunction, especially if there plenty of money floating around.
There is one other thing I can suggest to CEOs. Take a look at the earlier section in this article, "Dysfunctional projects: Paranoia, abuse, firing, screaming." Those are symptoms of a dysfunctional project. If you see those symptoms occurring, then you probably have a dysfunctional software project on your hands, probably heading for disaster. It's nice to know such things, but unfortunately, of course, there won't be a damn thing you'll be able to do about it.
(Comments: For reader comments, questions and discussion, see the 23-Aug-15 World View -- Fraud and subversion in Healthcare.gov - the greatest IT disaster in history thread of the Generational Dynamics forum. Comments may be posted anonymously.)