ABSTRACT
LifeLines provide a general visualization environment for personal histories that can be applied to medical and court records, professional histories and other types of biographical data. A one screen overview shows multiple facets of the records. Aspects, for example medical conditions or legal cases, are displayed as individual time lines, while icons indicate discrete events, such as physician consultations or legal reviews. Line color and thickness illustrate relationships or significance, rescaling tools and filters allow users to focus on part of the information. LifeLines reduce the chances of missing information, facilitate spotting anomalies and trends, streamline access to details, while remaining tailorable and easily transferable between applications. The paper describes the use of LifeLines for youth records of the Maryland Department of Juvenile Justice and also for medical records. User's feedback was collected using a Visual Basic prototype for the youth record.
Keywords
Visualization, history, timeline, personal record, justice, medical record, screen design, overview, screen management.
INTRODUCTION
Records of personal histories are needed in a variety of applications. Members of the medical and legal professions examine a record to garner information that will allow them to make an informed decision regarding their patient or case. Decision making critically depends on gleaning the complete story, spotting trends, noting critical incidents or cause-effect relationships and reviewing previous actions. Professional histories, in the form of résumés, help employers relate a prospect’s skills and experiences to employment and education. Financial and retirement plans associate past and upcoming events to culminate in an expected result. In child and family social services, complex problems may require review of multiple related personal histories.
In most applications delays in gathering the information to elaborate a meaningful overview of the record can have deleterious effects. In a medical situation, a patient’s treatment may be delayed while charts and lab results are assembled. In a social work situation, assistance to a youth
in detention may be delayed for weeks while school and court records are brought together.
Once gathered in a single record, the information is often in the form of a puzzle and the reader has to browse the data in order to form the big picture of the record. The reader must pour through lengthy and diverse pages. Missing or overlooking a piece of the puzzle can have dramatic effects.
While more attention is now put on developing standards for gathering and exchanging personal records (especially in the medical field), we found that virtually no effort had been made to design appropriate visualization and navigation techniques to present and explore personal history records.
An intuitive approach to visualizing histories is to use graphical time scales. The flexible and natural ordering of the time scale in months, weeks, days and even minutes, gives the design an efficient and facile interpretation and the ability to divulge detail. The consistent linear time scale allows comparisons and relations between the quantities displayed. Most importantly, large data sets can be displayed along the time line to help relate a story [15].
We propose LifeLines as a general technique for visualizing summaries of personal histories. On a one screen overview multiple facets of the records are displayed. Aspects with varying status, such as medical conditions or legal cases, are displayed as horizontal lines, while icons indicate discrete events, such as physician consultations or legal reviews. Line color and thickness illustrate relationships or the significance of events. LifeLines always begin with a one screen overview of the record, and rescaling tools or filters allow users to focus on part of the record and see more details, providing a general visualization environment. Techniques to deal with complex records and issues of a standard personal record format are discussed.
This general visualization environment is not computationally demanding, requires only high level data descriptions, and can handle a variety of records. Those characteristics make LifeLines a practical example of a personal record format that could be rapidly exchanged or synchronized between multiple services.
THE CHALLENGE OF PERSONAL HISTORIES
In order to be a general tool LifeLines have to be able to present different facets of a person's life (medical, financial, education, work, hobbies, legal, etc.). A particular application might use LifeLines for only a subset of those facets. Each facet includes different stories or aspects. For example the medical facet of a person's history might include a 5 years story about their back pain. A famous artist's biography might include facets on painting, writing and influential personalities, its painting facet would then have different aspects such as style and themes. Each story or aspect includes events (e.g. an operation, a police arrest) and periods (e.g. two weeks of acute pain, the blue and pink period in Picasso's biography).
LifeLines present the personal history in a single screen, facets are shown as regions of the screens distinguished by alternating background colors, stories or aspects are lines, periods correspond to changes of size or color along the line, while discrete events are marked by icons.
Our work on the visualization of personal history started with a project with the Maryland Department of Juvenile Justice (DJJ), a Visual Basic prototype was developed to illustrate the principle of LifeLines (Figure 2,Figure 3a, and Figure 3b), collect user feedback on the youth record design and explore the applicability of the LifeLines to other domains. Screen mockups have been prepared (Figure 4 and Color Plate 2) to discuss refined designs and particularly the use of LifeLines for medical records. .A video is also available in the CHI 96 Video [10].
RELATED WORK
Tufte [15] describes timelines as a frequent and powerful form of graphic design and presents many examples. An experiment in home automation showed that timelines could be quickly understood and used for data entry when scheduling devices in the home [10]. Novel techniques have been proposed to present calendars and timed information [9] and to analyze and search time scales [5] [14]. A design using timelines for medical records was proposed by Powsner and Tufte [13] who developed a graphical summary using a table of individual plots of test results and treatment data. In project management the Gantt chart [8] and PERT chart display the duration and type of tasks in a project. Many attributes of the tasks are stored but only synchronization and dependencies of tasks themselves are shown on the diagram. Software such as Microsoft Project uses these charts but the overviews do not provide the richness needed for personal histories, which are not carefully planned series of tasks but a mosaic of information that still need to be related to each other.
When navigating large spaces overviews have been found helpful [12] [2]. In the medical domain studies have shown the benefits of record summary. The Summary Time-Oriented Record, a pure text flowchart, was found to be helpful in an arthritis clinic [16]. Another text summary record uses scaled values to indicate the severity of a symptom and also notes the duration of the symptoms, allowing the timing of clinical events to become apparent [17]. Intensive care summary visualization software includes notions of filtering, severity scaling, and details on demand [4].
JUVENILE JUSTICE YOUTH RECORD
About 600 employees of the Maryland Department of Juvenile Justice are using an information system to process over 50,000 cases of delinquent youth behavior per year. In general a 'case' is opened when a complaint is filed against the youth (usually after a police intervention.) The success of a DJJ case worker’s decisions relies greatly on knowledge of the youth’s current status and previous case history. With the current system cryptic codes are used to access screens packed with lists of cases, placements or reviews, and to browse complex forms (Figure 1) while searching for detail information in dozens of screens. Important notes and decisions are often difficult to find in the deeply nested structure and more importantly it is very difficult to get an overview of the whole record. While attending a standard training session for the system we were told that the "trick is that you have to find the magic case", i.e. you have to be lucky and follow the right path to a case where a lot of information is available. The data entry is done over months or years, by clerical staff, field workers or detention facilities staff, geographically dispersed and with limited communication. Newly added or updated data may not be noticed, data entry errors are common and often remain unseen in the text screens. Delays are difficult to notice when browsing tables of dates.
In response the LifeLines were designed to:
Description of the interface
In this LifeLines design the facets of the youth records were chosen to be the cases, placements, case worker assignments and reviews (Color Plate 1 and Figure 2). Each facet is distinguished from the next by a switch of background color. The dates of the timeline legend shows that the youth Bart Simpson has been involved with DJJ for about a year. Thin vertical lines across the whole timeline mark each new year, giving a sense of scale to the diagram (e.g. compare Figure 2 , Figure 3a, and Figure 3b). The first facet of the record shows the cases. There are 4 lines indicating four cases in Bart's record. Bart was first referred to DJJ for "breaking and entering" (B&E code). The case was handled informally and then closed after 3 months. He was referred again for the same offense, also handled informally and then for a more serious auto theft. That time the case was handled formally (i.e. went to court, i.e. to the States Attorney's Office - SAO) and Bart was adjudged delinquent DLNQ (i.e. was found guilty). The case was closed several months later. The right end of the timelines area shows today's status: we can see that there is only one case currently open, an attempted murder case for which a decision has not been made yet. Below the cases the placement facet shows that Bart is currently at Waxter (a detention center), in relation to the attempted murder case. We can see that last year Bart went to a drug abuse program for a while and later was placed at Cheltenham (a residential treatment center) when he was found guilty in the auto theft case. The assignment lines show who has been assigned to Bart over time. The currently assigned worker is Brown. The last facet in this record is the set of reviews. Reviews are discrete events where the youth’s needs or progress are assessed. A click on Smith's name gives the worker's contact information. A click on a review icon brings the text of the review report on an overlapping window. Similarly details about cases or placement are obtained by clicking on the lines or labels.
Interrelationships between periods or events on the lines can be highlighted. For example one cannot tell by looking at the timelines if the drug abuse program was recorded as part of the auto theft case or the breaking and entering case, but clicking on the drug abuse program placement highlights the related case (i.e. in this instance the B&E case) and the social worker assigned to that case (Color Plate 1).
Line thickness and color are used to indicate the severity of the offense and the depth of penetration in the system (Color Plate 1). For example underage drinking would be a very thin line, but the alleged auto theft offense is a medium thickness line, which in the example of Color Plate 1 and Figure 2 is getting darker as the case progresses to court and darker again when Bart is found guilty (delinquent). On the other hand the recent case of alleged murder appears as a thick line but remains of light color because the case has not progressed yet. Severity levels are set following a policy.
At the top of the screen, next to the name, age and picture of the youth, a set of buttons gives access to the general contact information, aliases used, education, work and medical status screens but critical keywords from those screens always appear on the overview screen (e. g. suicide risk). Color coding could also be used for those buttons to suggest how recently the information was updated. A pale button can indicate 'out of date' data, a bright button recently modified information.
THE BENEFITS OF LIFELINES
LifeLines can:
DEALING WITH COMPLEX RECORDS
Keeping the overview
LifeLines begin with an overview of the entire record. Seeing this overview gives users a better sense of the type and volume of information available. Unfortunately, many youth records include so many cases that one page is not enough. Other applications such as medical records or biographies might require 50 or 100 lines. Screen size limits the number of lines which can be displayed, and the number of events and periods which can be identified on a line. Scroll bars are the common answer to pixel shortage but scrollbars are inadequate if not harmful - when presenting overviews as users often forget to browse the complete image, or worse are even unaware that part of the image is hidden. To guarantee complete coverage of the overview, detail must be sacrificed for completeness in the first viewing of LifeLines. This raises two issues for the designer: how to represent all the information at the overview level , and how to reveal the next level of detail. Those problems are general to the design of image browsers for monitoring and decision making [12], and we propose a series of techniques tailored to LifeLines.
Silhouette and shadow
When all the lines and labels cannot be shown in the first screen the lines are brought closer to each other and the labels disappear (Figure 3a and Figure 3b). It remains possible to see on this silhouette the number and importance of the cases, the type of placement and the number and date of reviews. Each line, period or event can still be selected individually. A background sensitive cursor can display the label of the underlying object (Figure 3a) and selecting a line or icon can display the labels of all associated events (i.e. the corresponding case, placements, assignments and reviews (Figure 3b)).
The natural next step for unusually large records is to remove all blank space and thickness coding, clumping all lines together to show a mere colored shadow of the record (which could then show hundreds of lines in one screen). Once the overview has been appreciated, semantic zooming can be used [18] as users expand desired facets to show thickness and labels again, zoom to a particular time period, or focus on a given case. Scrollable, resizable windows should only become necessary when the overview has been examined. Then filters can be offered to search, sort or hide items of a given type.
Facet hierarchy
LifeLines are organized by facets of the record. A complex record may require a hierarchy of facets containing the headings of the facets, stories and aspects. Displayed in the classic outline format (Color Plate 2), facets can be expanded and contracted to show increasing or decreasing levels of detail. Through direct manipulation, the user may rearrange the headings within any level of the tree. Sorting specified levels of the tree by importance, name, date or other factors becomes useful.
Rescaling
When the sheer density of periods and events on a time line requires intervals so small that they challenge the granularity of the time line a sub-time-line icon (or a special line style) can signify a section of the time line where the data density is too large. Details of that time period can be revealed progressively by rescaling (zooming in) or by opening a sub-time-line into a detail window. The term rescaling is preferred to zooming since only the horizontal dimension is modified. We were surprised by the diversity of design solutions to perform the time rescaling. Figure 2 shows a flexible-interval scroll bar, which is adjustable at both ends to represent the width of the desired interval of time, it offers adequate control, but requires the manipulation of a widget that is not altogether familiar. The screen mockup of Color Plate 2 shows compressed and stretched clock tools allowing the user to alternately compress or stretch the time scale centered on the last selected item. Another design is to separate the horizontal scroll bar from the selection of the time interval (e.g. week, month or year, which can be chosen in a menu or with a different slider). This is advantageous for reviewing specific fixed periods but requires users to focus their attention on the time period selection, rather than the data displayed. The 3D animation technique of the perspective wall [9] which folds the display in 3 panels could also be used but the facet hierarchy legend becomes an intrusion into the continuous scrolling. Lastly, the time scrollbar can be designed to allow multiple foci by dividing it into independent scrolling regions.
Layout issues
Optimized layout algorithms have not been explored in our project but the abbreviation and layout of the labels and compact layout of the lines is definitively a critical part of dealing with large records. We are not satisfied with the current prototype as too much space is left unused yielding a low "data-ink ratio" [15].
USER'S FEEDBACK FROM DESIGN REVIEW
The Visual Basic prototype was demonstrated to about 60 users during an all-day review. Small groups of 4 to 5 users were given a 20 minute demonstration of the LifeLines prototype. They were able to use the interface to explore a few record overviews that we had prepared based on real records. They were asked to comment on perceived advantages and problems, and to consider what additional data would be useful to display in the overview. Users were a representative group of social workers, supervisors and a few administrators of varying age, gender and computer experience recruited from field offices, detention facilities, program services and administrative services.
Most users were very enthusiastic about the interface. A few others appreciated the interface but expressed concerns about the possible bias associated with the color and thickness coding. Showing the offense severity brought to the surface underlying issues such as the data "quality" (possibility of errors or cases of uncertainty). The paradox might be that error prone data buried in a hard to navigate system seems less offensive than when the same data is made visible. The use of color and thickness coding is very powerful, and should be used carefully to avoid misleading overviews.
Positive comments were related to:
Data entry errors were found in the records we used, which would have been difficult to locate in the multiple screens of the current text record. The long tables of dates used in the current system are difficult to review, while the timelines make more visible the fact that a case was left open by mistake, that reviews are overdue or that the court is taking too long to make a decision.
Many recommendations were made:
LIFELINES FOR MEDICAL RECORDS
Because our youth record Visual Basic prototype reads a simple format as input it was easy for us to explore other data sets such as our own personal resumes and sample medical records, which lead to new designs and layouts shown in the medical record screen mockups of Figure 4 and Color Plate 2.
An appropriate use of LifeLines
A patient's medical record is an important source of information for the physician. Medical records can be extremely complex, with data intervals ranging from seconds to decades. Clinical data lend themselves to a multimedia presentations, with sounds, such as patient interviews; images, such as X ray images or MRIs; quantitative data, e.g. lab test results or vital signs; and qualitative data, such as visual observations.
Certain characteristics make medical records specially suited for the LifeLines format:
In the sample medical record (Figure 4), we see that the patient is diabetic from medical alerts section at the top of the page and from the line in the manifestation facet. For treatment, he has regular, bimonthly consultations with an endocrinologist and administers insulin to control the symptoms. Early in May, this patient sprained his knee and visited an internist, his primary care physician, for suggestions. Dr. Wood referred him to an orthopedist, Dr. Jones, whom he visited for an opinion on his knee. The patient, however, desired a second opinion and visited Dr. Hansen who suggested arthroscopic surgery and prescribed acetaminophen to relieve the pain. Shortly after, the patient entered University Hospital where Dr. Hansen operated on the damaged ligament. Having received surgical treatment, the knee proceeded to heal and the severity of the tear, implied by the thickness of the line, decreased and the dosage of acetaminophen also decreased.
After leaving the hospital, the patient began physical therapy sessions with Dr. Carter. As the ligament continued to heal, sessions became farther apart, with two more scheduled appointments past the current date marked by the dashed red line in this example. Future events occur to the right of the current date line and appear gray. Meanwhile, the patient developed ear pain and returned to his internist. The internist saw in the medical alerts that the patient is allergic to penicillin and prescribed the antibiotic doxycycline for the diagnosed ear infection. The ear infection quickly healed and the patient finished off all of the antibiotic.
Encoding information
Labels for the consultation events in Figure 4 denote the chief complaints of the patient. They may, however, contain other information, such as the resulting diagnosis, or a suggested course of action, for example, a referral to a specialist. Other icons may be more indicative of the nature of the consultations: A clipboard or writing hand may represent a purely observational consultation, a flask means that lab results are available, an ambulance suggests an emergency situation and a syringe indicates that medications were administered (Color Plate 2).
Showing relationships
Quantities sharing a critical quality are grouped into similar facets but it may be important to view relationships between items outside of facet boundaries. Through the simple and consistent use of colors, groups can be identified. For example, color can be used to signify lines and events related to a diagnosis listed in the manifestations heading. In the example printed in Figure 4 all information related to the sprained knee was green and all information related to ear pain was magenta. This is a very telling scheme. However, it may be crucial to locate every consultation, illness and treatment related to Dr. Wood. Color Plate 2 shows how each physician can be assigned a color to mark his or her consultations, letters, prescriptions etc. The list of doctors serves as the color legend for the diagram.
OVERVIEW AND DETAILS - ELASTIC WINDOWS
Assuming that a large screen is available multiple windows can be used to see detailed reports while keeping the overview (Color Plate 2). By dragging an event icon to the region, space is allocated for a formatted report of all information associated with the icon. As more icons are placed on the region, the region divides into independently scrolling segments, with the new report placed in relation to the other reports with respect to where it was dropped on the screen [7].
When large records have been expanded in scrollable facets windows and multiple detail windows, new methods are needed to manipulate them beside the traditional vertical scroll bar assigned to each region, 1) the pump tool, available from the tool bar in the upper left corner of the time line region (Color Plate 2) is used to expand or contract the region from both ends, shrinking the adjacent regions [1], 2) the hook tool expands or contracts the region from one end and shrinks the adjacent region [1]. The master vertical scroll bar on the right hand side of the LifeLines allows the user to navigate across all the top-level facets.
TOWARD A PERSONAL HISTORY RECORD FORMAT
In order to develop a standard for the display of personal histories certain concessions are necessary to maximize the chances of their use. The use of pointers to other data sources offers an alternative to the unsuccessful attempts at all-inclusive, redundant records. The knowledge of the existence and type of an event is in some cases all that matters, and in all cases better that not having any record of the event at all because the information is in an incompatible format. Because of its simplicity and tailorability the overview of the LifeLines could act as a referencing tool to a complete physical record, with immediate access only to high level data. As more detail is needed, a system using LifeLines can establish a network connection with the appropriate system if possible or use contact instructions stored in the LifeLines to avoid an irking “data not found” message.
LifeLines offer an effective visualization tool, but reality often thwarts complete and immaculate record keeping. Imprecise data should be acknowledged with adequate coding (e.g. an approximate date of birth should appear differently than a verified one). Confidentiality needs to be secured at least at the same degree as a paper record. The record owned by an individual would include a medical facet which can be copied when visiting a new doctor's office, providing basic summary data and pointers to other physician's records. But the patient would not authorize the doctors' office to copy bank account summaries or pointers to school records. Similarly social services could be authorized by the person or by the court to synchronize their records with court systems or school systems and speed up their required data gathering.
LIMITATIONS
For the designers, agreeing on an appropriate data encoding scheme is difficult. Icons, color and thickness codes have to be carefully designed to avoid biases. For the developers appropriate labeling of the timelines remains hard to optimize and smooth rescaling depends on efficient display algorithms, [6] [17].
Only the visualization aspect of the record has been discussed here and none of the data entry issues. Even if in many applications tedious data entry remains separated from the decision making task, it is clear that a complete implementation of the LifeLines will need some data entry mechanism to append or correct existing records. Our previous experience shows that constructing time lines by direct manipulation was an adequate solution in scheduling home automation devices [11].
CONCLUSION
Personal histories contain information that plays an important role in decision making. LifeLines are proposed as a simple and tailorable technique to represent a variety of personal history records. They provide a complete visualization environment offering overview, zooming, filtering and details on demand. Color coding, filtering and dynamic highlighting unveil relationships between events that may otherwise be difficult to see in paper records. Finally, techniques have been described to handle large records and to facilitate the associated window management, making LifeLines a useful starting point toward a standard personal history format.
ACKNOWLEDGMENT
This work was supported in part by the Maryland Department of Juvenile Justice and by the NSF grant NSF EEC 94-02384. We want to acknowledge Chris Cassatt from DJJ for his participation in this work, and Teresa Cronnell for her help with graphic design.
REFERENCES
plaisant@cs.umd.edu
