Document Driven v Data Driven

I’ve recently been thinking a lot about forms. Why forms? Forms give us fascinating insights into that way that organisations work.

A Life of Forms

We are surrounded, some would say inundated, by forms:

• Banking runs on forms.
• Insurance wouldn’t survive without forms.
• Most organisations have thousands of ad hoc forms for various diverse purposes.
• One of the worst things to happen in some organisations is that a situation arises for which there is no form.
• Visit a medical professional and somewhere within the dialogue a form will become necessary.
• Subscribe to any service and forms will be used as part of the contracting process.
• Start a new employment and you are likely to spend much of your first day completing forms.
• Our birth and our death are accompanied by forms.
• How many times a day do you complete a two-field form in order to gain access to some technology.
• Interact with a government organisation and a form will be required.

Sometimes these forms are online, web page, or even forms on mobile devices. There are still, however, many situations where forms are completed with a pen. How many hours have you spent trying to complete a pseudo form that was sent to you as a Word or PDF document.

Document Driven Business

There are many PowerPoint decks, Excel spreadsheets and Word documents that are in essence forms. They are created from a template that sets the titles and contents of each slide/worksheet/section. The person completing them is expected to say certain things in certain ways, just like a form.

• This first slide has the title on including the reference number, person presenting and target date.
• The next slide has the required content on and only this content.
• The following slide will explain what it is you are going to do.
• The penultimate slide will outline the business case in the supplied table.
• The final slide will contain the risk register, using the supplied table headings.
• No other slides may be added.

It’s a form, isn’t it?

A Form to Transact

Each of these form-types exist to support a transaction:

“Once you have completed sections 1 to 5 and 8 of the loan application form we will proceed to the next phase of you application.”

“You are required to complete a tax return of which sections a to e are mandatory.”

“We’ll proceed with your project once you have provided the project initiation template document.”

The boundary of the transaction is defined by the form, without the form nothing moves forward, or backward.

This way of working produces a number of effects:

• Over preparation – in order to make sure that a transaction can complete documents tend to be over-worked. Many hours are spent making sure that every detail in a form/document are correct to a level of detail that is not required to move onto the next phase, but everyone strives for perfection to avoid rework at all costs. A small amount of over-work is compounded as a process is worked end-to-end. Imagine how much work goes into producing a set of 40 document? Add a little bit of over-preparation to each of them and the amount of effort being expended is huge.
• Over-stating – The over-preparation of documents often includes over-stating, where things that aren’t required in the document are stated in the document “just-in-case”. The problem with this superfluous information is that it becomes part of the record and is then used by people who make decisions despite its heritage and trustworthiness.
• Point-in-time perspectives – The information in the form/document was mostly correct at a particular time on a particular day, but that’s all that can be said about it. Any perspective that is taken on that document is locked into the context at that time. The information in the document isn’t being refreshed, it was completed, a transaction took place and now everything within the document is, at best, history. Yet, people will continue to refer back to it as information way beyond the valid life of the data contained within it. The reality is, even before the document is concluded the data within it will be out of date.
• Action blocking – A form/document represents the end of one activity and the start of another – a phase-shift. The next phase can’t start until it has received the information from the previous phase. Even if an element of the next phases has all that it needs to proceed it can’t until the transaction has been agreed. Consider how many actions are expected to be undertaken following the transaction of a 100 page document? How many of those actions could have safely been undertaken way before the transacting of the document?
• Phases based on documents – The definition of a document as the point of transaction means that production of the document often becomes the definition of the phases/stages of an activity. This way of planning has little to do with the amount of effort involved, or the value being produced, it just represents a transaction. An activity that only exists to produce a document is a bad activity.

Data Drive Business

Let’s turn our attention to data-driven activities.

The document has been with us for thousands of years, but we no longer need to work at such a coarse level. The information that is placed into a form was not generated by the form. A form is just a place to consolidate information that already exists elsewhere. When you are asked about your date-of-birth in a form you are simply recording information that has existed, for some of us, for many years. So why not link the data directly with the intent for which it is needed. Why bother placing a date of birth on a piece of paper when one system could ask another system whether I’m older than 18 and get the correct answer back.

There are situations where data isn’t enough and a set of information may need to be brought together to tell a particular story. Imagine a design for a network topology, the design may be the first time that it’s been outlined. This isn’t to say that in this situation a document is required, it’s just to highlight that an intermediate step from current state to future state may be required to fill gaps in the data. Even in this network topology example a diagram with meta-data is probably sufficient to communicate the change being proposed and for people to agree to transact. Once the change has been implemented the diagram is no longer required because the current state information becomes the record.

Taking the network topology example even further, the need for a human-readable design demonstrates a gap in policy and understanding. If the change could be codified in a way that a policy mechanism could understand and assess, then the change could have taken place without the need for a diagram. If, as an example, an application needs to add more resources to the network, the network would respond on the basis of the data provided and the policy defined. Likewise, once those resources are no longer required the policy engine would turn the resources off. All of this would happen before someone has filled in half of the “necessary” paperwork.

Our job, as humans, should be to assess and define the required actions for the exception, for those situations where data and policy is not sufficient for a decision to be made.

Time for Transformation

For much of the life of IT may applications have been little more than form replacements and that has given us some productivity gains. In many ways we are only just at the beginning of a transformation from a world driven by documents to one driven by data  This will require a profound change in the way that we think and act.

Organisations that continue to rely upon forms (including apps that are replacing forms) will be overtaken by the machines.

Header Image: This is one from a recent morning walk along the lanes near my house. I’ve always loved the shapes of tree skeletons in the winter.