Archive for the ‘Standards Development’ Category

Judging Web Site Quality: Combining Objective Tools & Collaborative Filtering

Wednesday, May 30th, 2007

In response to a column in The New York Times last week (http://www.nytimes.com/2007/05/24/fashion/24Cyber.html), I wrote the following letter to the editor (for which they apparently aren’t publishing any letters):

Regarding “Visits to Doctors Who Are Not in, Ever” (May 24), democratization of information has made health content widely available—some would describe this a blessing and others a curse. The question is how do we steer people to information that is accurate, comprehensive, relevant, understandable, and useful?

Empirical research we’ve conducted on diabetes Web sites suggests that the guidance offered by most “experts”—such as a “trustworthy” sponsor, the currency of the content, or the process for updating it—provides little insight into the content’s actual accuracy or comprehensiveness.

Innovative collaborative filtering approaches can be effective, but they also have limitations—in part due to the complexity of medical science and its pace of change. Objective, systematic instruments now exist that could be scaled to provide critical guidance to consumers. These two approaches can complement each other and help people navigate life-and-death matters for themselves and their loved ones.

Since I get more than 150 words here, I’ll expand a bit…. First some quick background: The research I refer to is summarized in a white paper on “The Mysterious Maze of the World Wide Web” on the www.ixcenter.org Web site and the more scientific papers are available from the peer-reviewed Journal of Medical Internet Research (www.jmir.org).

It’s reasonable to approach the evaluation of Web site information quality like many other scientific questions where the answers (to “what is high-quality content?”) are not always straightforward. We can triangulate by using multiple methods to answer the question.

Specifically–and as I proposed in the original research–we can combine three kinds of measurement: structural measures of quality to give a generic content threshold test; performance measures of accuracy and comprehensiveness for specific conditions; and assessment of consumers’ perspectives on content in terms of functionality, understandability and overall utility.

In my original work (before the “Health 2.0″ space existed) , I had envisioned this being done more though random consumer survey methods, but the collective filtering approach is considerably more efficient, even if less scientific.

Bringing these different strategies together will provide everybody with a lot more guidance about health information quality, the first ingredient for good information therapy.

–Josh

MyHealth, Circa 2007: Consumer Needs and Market Responses in eHealth

Tuesday, February 6th, 2007

We held a public webcast on this topic today. Susannah Fox, Associate Director of the Pew Internet Project, not only shared valuable insights but also some of Pew’s as-yet-unpublished data. The presentations and the audio recording from both of our presentations will be available on our Web site in the near future.

We had nearly 100 unique logons to the webcast and more than 25 questions, so we didn’t have time to answer all of them. Over the next few days, I’ll try to answer questions we didn’t get to and elaborate on a few that I have been thinking about for various reasons. Also, Susannah Fox will post comments on this blog (as time allows) as well…although I’m sure that you’ll need to tune in to her upcoming Pew reports (available at www.pewinternet.org) to get all her insightful thoughts.

Q: Why can’t we have a way for doctors to upload my personal health records to a private page on a social network site that I can then share with other providers or people in my community?

JJS thoughts: This is a great idea, and clearly an activity that would support a much more collaborative care environment that would help improve (to paraphrase Dartmouth’s John Wasson) “same-page care” (getting clinicians and their patients on the same page). Unfortunately, there has been little progress in this arena outside of some integrated delivery systems (perhaps Group Health Cooperative in Seattle being the most advanced example). Some of the organizations that Wasson works with through a couple of innovative initiatives (see www.howsyourhealth.org and www.idealmicropractice.org) are getting at this in different ways as well.

No doubt part of the reason is that an infrastructure doesn’t exist to support it, and we’re working on two aspects of that infrastructure. First, we need to address the consumer-facing side of IT interoperability so that we can effectively connect consumers to health content that they can make sense of. I wrote an Issue Brief for the California Health Care Foundation about this issue (see “Lost in Translation: Consumer Health Information in an Interoperable World” at http://www.chcf.org/topics/view.cfm?itemID=114624).

The second infrastructure piece that we are trying to address is the payer side of the equation by creating technical specifications to define what constitutes Ix. The goal of our Ix Payer Workgroup is to provide guidance to pay-for-performance, provider recognition, and other programs so that they can recognize when providers, organizations, or tools provide people with information that supports their information needs with respect to decision support, behavior change, or other health needs.

I’ll address other questions tomorrow and in the days ahead.

–Josh

Making the Case for Information Therapy (Ix): Recognition, Reimbursement, and Research

Tuesday, January 9th, 2007

Since research* suggests that 50% to 80% of everything that a patient hears in the doctor’s office has been completely forgotten by the time he or she gets home, it remains remarkable to me that payers don’t require an Ix after-visit summary as a condition for reimbursing clinical encounters.

For reasons such as this, the inherent logic in Ix is so compelling from both a practical and moral standpoint for many of us. However, most of the payer community is not ready to accept that logic without a substantial body of research behind it—not that any empirical evidence exists for the value of the generic clinical encounter itself.

To be fair, we first need to define in specific terms what constitutes Ix if we think that the market should recognize it. Indeed, that is the first task before the new Ix Payer & Certification Issues Workgroup, which was launched at the end of November as part of the IxAction Alliance.

The workgroup is chaired by Eleanor Herriman, MD (Chief Science Officer of Boston-based NorthPoint Domain) and includes a diverse cross-section of organizations. The Ix Payer Workgroup not only recognized this definitional need, but also realized that not all Ix is created equal—it varies both by type and level of complexity—and therefore has embarked on the more ambitious agenda of developing an Ix taxonomy.

As we build the infrastructure for identifying when and how Ix is performed, we also need to continue to build the empirical evidence base for Ix to demonstrate the value that high-quality Ix brings to people’s health. That work is also a critical part of the IxCenter’s 2007 agenda.

One of the challenges in measuring the impact of the latest Ix innovations is that the leaders often don’t have time to design rigorous, prospective studies. They just forge ahead and innovate because they know it’s the right thing to do for their patients, members, or consumers.

That poses challenges to health services researchers (and I admit to being one myself :-)), who generally would prefer to have things evaluated more cleanly or traditionally. It’s harder to tease out cause and effect when studying innovation that transpires organically or as part of lean operational development processes.

However, we should not shy away from the empirical idiosyncrasies that innovative care delivery initiatives create. Rather, we should rise to the challenge by employing a broader set of research and analytical skills to tackle these compelling research questions about new innovations. Indeed, the new care delivery strategies create opportunities for health services researchers to develop their own innovative research techniques.

I hope that health services researchers out there are up to that challenge.

–Josh

*For references on this and related research, see the IxCenter’s white paper, “The Ix Evidence Base: Using Information Therapy to Cross the Quality Chasm,” which you can download from the Publications section of IxCenter’s Web site (www.ixcenter.org).

Patient-Centered Health Information Technology in the New Congress

Wednesday, January 3rd, 2007

With the 110th Congress about to get underway, it’s exciting to think about the possibilities for patient-centered health information technology (HIT) legislation. Last fall, Congressman Patrick Kennedy introduced a bill (HR 6289) that would create incentives for using personal health records (PHRs). Unlike much of the HIT legislation out there, it really focuses on consumer’s personalized information needs and enhancing the clinician-patient relationship. In particular, the bill describes requirements including that a qualifying PHR has the capability of “sending patient-specific patient education, reminders, and clinical messages to patients based upon data in the record.”

The bill, appropriately titled the Personalized Health Information Act, that was introduced in the 109th Congress likely will be reintroduced soon for the new Congress. Although it’s not clear how strong the financial incentives will be (though private & public organizations can contribute to the incentive fund with targeted information therapy (Ix) campaigns), there are several sections that are all about Ix…without calling it that.
For example,

  • One of the provider requirements is that (Sec. 2.c.4) “The physician (or provider) uses the QPHR as appropriate and authorized by the patient to communicate appropriate patient education and care management messages.
  • And the entire section (2.e.1.E) on “Messaging Capabilities” is all about Ix…

E) MESSAGING CAPABILITIES-
(i) EDUCATION REMINDERS- Subject to clause (v), the record is capable of sending patient-specific patient education, reminders, and clinical messages to patients based upon data in the record, but such messages shall not be sent unless such messages comply with standards adopted under paragraph (3). The Secretary shall work with the Secretary of Homeland Security and the Director of the Centers for Disease Control and Prevention to optimize the public health and emergency response capabilities of the networks created by QPHRs.
(ii) FEDERAL REMINDERS- Subject to clause (v), the record provides for the sending on behalf of Federal agencies of objective, accurate, patient-specific messages to patients concerning their health care or benefits, but such messages shall not be sent unless the messages comply with standards adopted under paragraph (3).
(iii) FUND PARTNER MESSAGES- Subject to clause (v), the record provides for the sending, on behalf of Fund partners who contribute to the Fund, appropriate patient-specific messages to consumers (with whom such partners have pre-existing relationships) concerning the patients’ health care, medications, treatments, medical devices or benefits, but such messages shall not be sent unless such messages comply with standards adopted under paragraph (3).
(iv) HEALTH PLAN NOTIFICATION- The QPHR service notifies, no less frequently than quarterly, each Fund partner that administers a health benefit plan of the individuals who are enrolled in the plan and who have a QPHR established.
(v) LIMITATION ON COMMERCIAL SOLICITATION- The record does not allow any commercial solicitations, marketing, or messages to patients unless the patient is a patient or beneficiary of the sender, uses the sender’s product with a prescription or recommendation of a provider, or has some other pre-existing relationship (as defined by the Secretary), or other messages that do not comply with standards adopted under paragraph (3), and the record ensures that every message clearly identifies the source of the content.
(vi) PATIENT OPT-OUT- The record allow a patient (or patient’s authorized representative) to opt out of receiving messages entirely or from particular sources.
Honestly, I don’t have any idea what the chances are for this legislation to move forward, but at least it’s nice to know that HIT legislation is being proposed to address the ways that HIT can truly support patient and consumer empowerment.

Honestly, I don’t have any idea what the chances are for this legislation to move forward, but at least it’s nice to know that HIT legislation is being proposed to address the ways that HIT can truly support patient and consumer empowerment.

Let’s hope that this is just the beginning.

- Josh