Archive for May, 2007

Judging Web Site Quality: Combining Objective Tools & Collaborative Filtering

Wednesday, May 30th, 2007

In response to a column in The New York Times last week (http://www.nytimes.com/2007/05/24/fashion/24Cyber.html), I wrote the following letter to the editor (for which they apparently aren’t publishing any letters):

Regarding “Visits to Doctors Who Are Not in, Ever” (May 24), democratization of information has made health content widely available—some would describe this a blessing and others a curse. The question is how do we steer people to information that is accurate, comprehensive, relevant, understandable, and useful?

Empirical research we’ve conducted on diabetes Web sites suggests that the guidance offered by most “experts”—such as a “trustworthy” sponsor, the currency of the content, or the process for updating it—provides little insight into the content’s actual accuracy or comprehensiveness.

Innovative collaborative filtering approaches can be effective, but they also have limitations—in part due to the complexity of medical science and its pace of change. Objective, systematic instruments now exist that could be scaled to provide critical guidance to consumers. These two approaches can complement each other and help people navigate life-and-death matters for themselves and their loved ones.

Since I get more than 150 words here, I’ll expand a bit…. First some quick background: The research I refer to is summarized in a white paper on “The Mysterious Maze of the World Wide Web” on the www.ixcenter.org Web site and the more scientific papers are available from the peer-reviewed Journal of Medical Internet Research (www.jmir.org).

It’s reasonable to approach the evaluation of Web site information quality like many other scientific questions where the answers (to “what is high-quality content?”) are not always straightforward. We can triangulate by using multiple methods to answer the question.

Specifically–and as I proposed in the original research–we can combine three kinds of measurement: structural measures of quality to give a generic content threshold test; performance measures of accuracy and comprehensiveness for specific conditions; and assessment of consumers’ perspectives on content in terms of functionality, understandability and overall utility.

In my original work (before the “Health 2.0″ space existed) , I had envisioned this being done more though random consumer survey methods, but the collective filtering approach is considerably more efficient, even if less scientific.

Bringing these different strategies together will provide everybody with a lot more guidance about health information quality, the first ingredient for good information therapy.

–Josh

Ix Across America: Integrating Information Therapy Into Practice Community by Community

Wednesday, May 23rd, 2007

We are eager to identify new strategies for helping practicing clinicians to make it easier to integrate information therapy (Ix) into their clinical workflow. One model that we have conceptualized is collaborating with leading local and regional quality leaders to stimulate Ix advancement in their own communities.

We are aware that this “Ix Across America” concept requires not only a combination of a lot of intense work and patience on our part, but also strong leadership from respected champions in each partnering community. I’m happy to report that our first test of this concept is off to a great start, thanks to the leadership of Helen Riehle, Executive Director of the Vermont Program for Quality in Health Care (VPQ), and several other committed innovators in the small but creative state of Vermont.

The report below summarizes the first phase of our “Ix Across America” best test in Vermont.

VPQ devoted its 2007 Spring Learning Forum to information therapy (Ix®). VPQ has helped diverse medical practices across the state to implement learning collaboratives, particularly around Ed Wagner’s Chronic Care Model (CCM).

After some introductory comments from the VPQ Executive Director and Vermont’s Lt. Governor, I set the context by explaining Ix in detail and contextualizing it vis-à-vis the CCM and related quality improvement (QI) strategies. Ted Eytan, MD, MPH (family practice physician and Medical Director, Health Informatics, Group Health Cooperative), presented many specific Ix initiatives Group Health has implemented in the context of its broader patient-centered informatics strategy.
Rushika Fernandopulle, MD, MPP (Internist and Co-Founder, Renaissance Health), described how he has used Ix in a small practice. In both cases, these practicing physicians provided considerable detail regarding the challenges and solutions for integrating Ix into the clinical workflow.

Forum attendees separated for breakout sessions. Practicing clinicians, practice managers, hospital administrators, and others affiliated with specific clinical practices discussed opportunities for applying these Ix approaches to their own settings. Attendees representing state government agencies, quality organizations, and those representing Vermont Information Technology Leaders (VITL, the state’s regional health information organization or RHIO) engaged in a policy-oriented discussion of how the state could facilitate greater Ix adoption.

When the breakout groups reported back to the full audience after lunch, these robust discussions resulted in several concrete next steps for Ix advancement including: implementation of multiple specific Ix initiatives in different practices; incorporation of Ix and consumer-focused activities into VITL’s pilot projects; integration of Ix components into the Vermont Department of Health’s Blueprint for Health project; and other statewide collaboration opportunities. The most popular specific Ix initiative that
Vermont practices plan to implement is the after-visit summary, but clinicians and administrators plan to pursue several other initiatives as well and there is broad interest in personal health record implementation. With respect to VITL and the Blueprint for Health, leaders believe that the architecture that is created can be used to make Ix a more explicit goal. In addition, VITL’s first implementation project related to medication history could involve components that more directly support consumer engagement in their health.

In the afternoon, Bob Corrigan (Vice President, Product Management, Emmi Solutions) discussed the “power of conversation” and demonstrated an interactive, engaging approach to Ix via text, audio, video and other graphics. All of the day’s speakers and state health care leaders involved in VITL and the Blueprint for Health (Cy Jordan, MD, MPH, Medical Director, VPQ; Andrea Lott, CIO, Northeastern Vermont Regional Hospital; and Jeremiah Sable, MD, Public Health Physician, Health IT, Vermont Department of Health) participated in a panel discussion to identify the greatest opportunities for integrating Ix and consumer-facing HIT applications into Vermont’s critical statewide health care initiatives.

Both Ted and I were able to frame all the day’s discussions after spending the day before in individual practices in
Vermont and with leaders from VITL and VPQ. These visits provided context for the specific challenges unique to
Vermont clinicians and HIT leaders.

We look forward to following up with Vermont practices, hospitals, HIT leaders, and government agencies through several concrete next steps to ensure that HIT implementation involves consumer-focused initiatives. Other quality leaders who think their communities would be good Ix laboratories should let me know.

–Josh