I hate software demonstrations. They are nearly useless. Here are my primary gripes…
Suspension of disbelief
I cried when I saw ET. That is the magic of movies. Somehow a perfectly rational person can believe that a lost alien has been rescued by children. I think that same aspect of human nature betrays us when we watch software demonstrations. People want to believe. I have seen people swear that they saw something in a demonstration that I know was impossible. Somehow, people see what they want. Perhaps people are too optimistic.
“Cannit”
I know I am trapped in a bad demonstration when a volley begins between participants and demonstrators. Each question begins with “Can it…” Of course each response is “Yes.” Or my favorite “Yes, with customization,” which is vendorspeak for NO.
There are so many problems with this approach I don’t know where to begin. Actually I do. The vendor is lying through his/her teeth until proven otherwise. Any simple question that someone asks can be interpreted in a way to illicit a positive response. Usually the questions are too poorly thought out to really capture the intent. Furthermore, if the vendor is not demonstrating, but just volleying back positive responses is anyone really learning anything?
Watching TV
I guess Americans would rather watch TV than read a good book. That is the true when it comes to the software selection process too. Everyone would rather crowd into a room to see a fraction of the functionality demonstrated than read through the documentation to really learn how the system works. See my earlier post about reading documentation.
Why are we here?
When we do have software demonstrations (which is as seldom as possible) I always make sure we start the meeting by telling everyone why we are gathered. There are two reasons for demonstrations: education and acquisition. Everyone assumes that a software demonstration will lead to a purchase decision, so it is important to strongly emphasize to all parties if you are just window shopping.
It is OK to have software demonstrations just to spark ideas and expand your understanding of what is possible. When that is the case it is critical that the participants and vendors know this and that the message is clear that there is NO commitment beyond today. These demonstrations should have a very limited internal audience.
If the demonstration is part of a selection process at my organization that will be self evident. The demonstrations will be tightly scripted. Everyone participating will have assignments, including:
- completion of evaluations that are tied back to the features/functions required to achieve the project goals;
- documentation feature/functions that need to be pursued; and
- identification of possible gaps.
Bottom Line
If there is no written documentation that is attached to the contract than this whole exercise has limited value. There is no obligation on the vendor’s behalf to provide the client with anything discussed during a demonstration.
Candid CIO 
Will, what are your thoughts about vendors sharing their architecture, design, and database models? Have you ever asked for them? Were they shared with you? I find that in most of the time when I’m consulting for clients I tend to ask for that material but the client doesn’t seem to think it’s important to get it from the vendor so the vendor shrugs it off. I agree with your request for documentation and I was wondering what success rate you have when you do ask for it.
Regarding wanting to see a demo rather than read the doc to find out what the product can actually do, I think you’re being a bit harsh. Frequently the doc does a lousy job of telling you what the product CAN do; it just is intended (sometimes in excruciating detail) how to make it do what it can do. Its not an easy position to be in. I just installed a product for a customer and finally, in desperation over being unable to get useful info from the PDF, paid to have the whole manual printed off. I’ve only read about twenty pages, but already found two things that are useful, that I didn’t know it could do. Is it in the doc? Obviously. Is it obvious? Nope.
I agree completely regarding demonstrations. They are a waste of time – PERIOD. Also, the RFI/RFP process is a waste as well unless you tell the vendor that any answers placed in the document will also be placed in the contract and implementation project plan. But, even with those steps the time involvement and lack of solid answers leaves it a questionable step. I like to get a listing of reference vendors/partners, etc. and take my time talking with them. Much like an interview. Also, having the reference hospital do the demonstration/presentation without sales staff present is great! After all this then I would dual negotiate with the top two vendors and each would know who they are up against. I have actually been a part of searches where the CIO took a RFP/RFI off the web and it seemed like the answers were never used. A serious waste of time for all parties. A presentation or demonstration can be a good tool to generate interest in the different business units, but it is not a good selection step.
Will, I want to clap with one hand and cry with the other. In one of my previous positions as a project manager with HBOC our sales staff gave extensive demos on vaporware that was scripted with only the working code of the applications. They sold 100s of clients before we had a fully functional product. When it was handed off to me to manage the project, the majority of my time was spent resetting expectations. I eventually left the company because I couldn’t be a car salesman anymore. I still see it today with companies like Omnicell and Cerner. The directive of the enterprise vendors is to sell at any cost. My current company demos and sells products that work. Fortunately or unfortunately, depending on how you look at it, we remain a relatively small vendor. The industry’s expectations have been set so low and there is so much distrust that I still experience the same issues with my current clients. They simply do not trust me or the product until we deliver on what we say we will. Companies like Epic are riding a wave because their service is one step above mediocre….It’s sad. My suggestion: put the RFP in the contract, put the project plan in the contract, put the application documentation in the contract, hold vendors to well defined milestones and patient outcomes, and hire a fighter as your project manager. I always use the analogy that healthcare IT is a model T running on bad fuel. The only place to go is up.