Data flows: Note on Data-Driven Process Modeling

A Focus on Data
Data flow diagrams (DFDs)
Common DFD mistakes
Summary/next steps


Since at least 1960, systems analysts have faced the task of describing business processes in order to build information systems that improve the performance of the business units in which those processes occur. This is true whether the analyst is designing a system to automate a structured task (such as order entry) or to support much less structured objectives (such as business strategy formation).

Unlike other types of analysts and consultants, systems analysts have not had the luxury of modeling a process and then walking away: the information systems they design have to function day in and day out as successful process support. Unsuccessful systems are often exposed as dramatic and expensive failures; they can even become lightning-rods for the kind of organizational blame that no one would want associated with their career. For these reasons, systems analysts have put considerable effort into developing ways of understanding processes that can translate directly into information system specifications. These specifications, often demanding detailed descriptions of database structures, records, and fields, are explicit and structured: as we saw with entity-relationship modeling, the constraints of computing and database technology demand clarity and precision in describing the data with which software applications will work.

Part of the systems analysis challenge arises from this mismatch between the knowledge that most organizations have of their internal processes and the kind of knowledge that a computer system needs in order to accomplish useful work. From a systems design point of view, processes are difficult to describe, for at least the following reasons:

  1. Processes tend to be understood differently by each person working within them (ask three different people how "the process works"; get five different answers).
  2. Processes sustain many variations. An order entry process, for example, may work one way for first-time customers, one way for large customers, one way for small customers, one way for former customers, and one way for customers whose purchasing agents are personal friends of your CEO's spouse.
  3. Processes can prove very complex. Again, an order entry process that seems quite simple at one level (e.g., I tell you what I want, you send it to me) proves very complex in large organizations that operate through many functional departments spread across a wide geographical area (e.g., you send a purchase order, which goes to an order entry clerk, who produces an internal purchase order, which goes to Shipping, where a warehouse assistant writes a shipping manifest, which gets sent to Accounting, where it gets matched with both the original and internal purchase orders, etc., etc.).
  4. An existing business process is not guaranteed to be optimally effective. This is a long-winded way of saying that not everything in every organization works as well as it was intended to (big surprise, right?). This statement of the obvious highlights another, perhaps less obvious point: sooner or later, most managers look to information systems as a tool for fixing problems that exist within current processes. In other words, they look for ways to use systems to redesign processes. Sometimes this is as simple as using a systems development project as an excuse for changing how an organization does its work [1]. At other times, the system development project itself will suggest specific ways in which to redesign a process.

The management implications of systems design arise largely from the interaction of systems design and process design - e.g., from an in-depth understanding of how information systems can be designed to foster constructive change within organizational processes. Each influences the other: in most systems design projects, processes influence systems and vice versa. Understanding these issues at more than a superficial level can greatly improve the success of any business design, process design, or system design ideas that you have.

Most traditional system design techniques do not arise from perspectives that managers might typically use to describe a business. Instead, they developed from a systems engineering perspective: techniques were evolved by software engineers who were trying to understand business processes in order to build information systems that worked (e.g., that actually got orders, shipments, and bills out the door without mistakes).

This point also might seem obvious, at least until you try to build a systems prototype of any meaningful size. In such a project sooner or later a "management" perspective (e.g., a focus on the business demands associated with the process in question) will begin to conflict with an "information systems" perspective (e.g., a focus on the demands imposed by the technology that information engineers are trying to use to support the process). The business demands of the process (e.g., to deliver a product or service quickly and without error, based on often highly ambiguous customer preferences) tend to oppose the requirements of an information system (which can process data extremely quickly, but must receive that data in highly-structured, unambiguous form).

The effect of these conflicting perspectives is illustrated below in Figure 1. It suggests that many characteristics of existing business processes can be left ambiguous, especially with regard to how those processes achieve explicit business goals. For example, it is often thought of as good management technique to give one's subordinates sufficient autonomy to work out operational problems however they wish so long as performance of their business unit achieves the strategic goals set for it by an organization's strategic plan: doing otherwise is often thought of as "interfering" or "micromanaging" or otherwise threatening the "empowerment" of workers. Note that such practice builds in a layer of ambiguity between strategic goals and operational tasks: senior managers are saying, in effect, that they don't care how the work gets done just so long as targets are met.

Information systems inherently micromanage: the technology constraints imposed by hardware and database design make it impossible for them to do otherwise. It sounds overly obvious to point out that databases must have fields and those fields must be defined - but when information systems are being used to support cross-functional processes that extend beyond any but the most operational levels, the impact of such requirements forces systems analysts to reach a level of process description that many managers never attempt. The input, processing, and output requirements of information systems drive a need for understanding process characteristics at multiple levels and in great depth.

From this point of view the success of systems design hinges on finding techniques that effectively reconcile an organization's naturally ambiguous understanding of the processes that achieve strategic goals with an information system's naturally structured approach to handling data. One approach to resolving this conflict involves understanding the structures inherent in business data, and gave rise to entity-relationship modeling. Another complementary perspective focused on how data moves through the tasks that make up business processes. This approach has come to be called data flow analysis, and is supported by a technique referred to as data flow diagramming.

Next: a focus on data

@ 1999 Charles Osborn