2 Methods

For tech policy analysis and design in pedagogical settings, we have found it useful to employ methods from value sensitive design. Friedman and Hendry (2019) give a comprehensive review of seventeen methods in value sensitive design, methods that have been used in diverse research, information system design, and engineering projects.

We do not reproduce that review, nor do we present a comprehensive tutorial. Instead, we present simplified introductions to the methods used in the case studies. These methods include:

• Direct and indirect stakeholder analysis
• Value source analysis
• Co-evolution of technology and social structure
• Value scenarios.

Direct and Indirect Stakeholder Analysis

Stakeholders are individuals, organizations, institutions, and societies. Stakeholders can also be non-human living things with moral or ethical standing such as salmon, killer whales, rivers, and ecosystems. In some projects, historic buildings or sacred mountain tops might also be considered stakeholders.

When a stakeholder interacts directly with a tool or technology they are referred to as a direct stakeholder. Direct stakeholders are commonly called “users.” In user-centered design, the aim is largely to improve the interaction between user and system, often as defined by usability assessments.

Indirect stakeholders, on the other hand, are not users—they do not use the technology under consideration—but are nevertheless impacted by it. Consideration of the interests of indirect stakeholders often lead to requirements that would otherwise not be identified.

For example, consider a patient’s electronic health record. Suppose a system is designed whereby medical personnel and insurance companies have access to health records but patients do not. In this case, the doctor and insurance company would be direct stakeholders and the patient would be an indirect stakeholder.

Putting aside the important question of whether it is reasonable to not give patients direct access to their health records, note that the patient has a tremendous stake in the system. Errors in the record might lead to diagnostic or treatment errors. Or, a data breach might lead to a patient’s privacy being compromised. In both cases, the possibility of physical or psychological harm is likely. Thus, indirect stakeholders can be as important to the success of a system as direct stakeholders.

Method panel 1. The outcome of a simplified direct and indirect stakeholder analysis of an Electronic Health Records system. (Suggested activity: Explore the activities and tasks that some of the stakeholders might need to complete with an Electronic Health Records system. What are the key communication and information flows among stakeholders?)

One common indirect stakeholder is the “bystander,” who unexpectedly encounters a technology and is impacted by it. Some examples:

• A street photographer inadvertently takes a photograph of a bus commuter (technology: camera; direct stakeholder: street photographer; indirect stakeholder: bus commuter, as bystander)
• A group of killer whales is adversely impacted by boat noise (technology: boat; direct stakeholder: boat captain; indirect stakeholders: the whales, as bystanders)
• A local teenager explains to a tourist how to connect to a community free wireless network (technology: wireless network; direct stakeholder: tourist; indirect stakeholder: teenager, as bystander).

A second common, indirect stakeholder is the “data subject,” a stakeholder for whom data is collected. Examples:

• As individual shoppers search and buy things, the resulting timestamped data traces (e.g., characters typed, mouse movements, links clicked, pages viewed, ads seen, etc.) are saved in data systems (technology: data analytics; direct stakeholder: business; indirect stakeholder: shopper, as data subject)
• Automatic license plate readers capture the location, time, and speed of a car as it moves through a city (technology: public cameras; direct stakeholders: police; indirect stakeholders: drivers, as data subjects).

Among the first steps of a tech policy project is to identify the direct and indirect stakeholders. Initially, for an in-class activity, this can be done by creatively exploring a problem situation in a short brainstorming session. For a substantial project, students might conduct a review of the scholarly literature, the popular press, or online message boards. Or, they might conduct an empirical study, perhaps semi-structured interviews with stakeholders or perhaps other fieldwork that draws on empirical methods from social science or engineering. While the case studies do you include robust studies in the field, they might be adapted to do so.

In the early phases of a stakeholder analysis, it is common to develop a long list of direct and indirect stakeholders. Accordingly, for the case studies, students are prompted to prioritize and select a small number of stakeholders and to explicate principled reasons for their choices.

Value Source Analysis

Human values shape and provide justification for tech policy choices. In value sensitive design, a value is defined as: what is important to people in their lives, with a focus on ethics and morality (Friedman & Hendry, 2019, p. 24). Examples of values include human dignity, justice, environmental sustainability, privacy, access, security, public good, safety, usability, calmness, fun, solitude, among many more. Many values may come under consideration in a project, and the relationships among values can be intricate. The set of values is often represented as a web or network, inviting discussion and clarification of the relationships among values. Commonly, when one value is addressed, other values become implicated. Security, privacy, and trust, for example, are often in an intricate balance.

When working with values, a key objective is to be clear and transparent. Conducting a value source analysis is one method for doing so.

A first step is to consider the values of the design situation and who among the stakeholders holds a value or set of values. For relatively short design activities, like the direct and indirect stakeholder analysis (see Method panel 1), students rely on their intuitions for the design situation and stakeholders.

A next step is to propose working definitions for the values under consideration. Working definitions orient designers toward the design situation and stakeholder interests, harms, and benefits.

Then, it is common is to clarify the origins of the identified values; that is, addressing the question of who proposed that a value be considered in the design process. Three common sources of values are:

• Explicitly supported project values. These are the values that are used to guide design processes. Explicitly supported project values can serve as design constraints at the beginning of a project or evaluation criteria for the finished system.
• Designer values. These refer to the personal or professional values that a designer brings to a research or design project. Environmental sustainability, for example, might be a designer value, but not an explicitly supported project value.
• Stakeholder values. These refer to the values held by different stakeholder groups. Frequently, stakeholder values are elicited through empirical investigations or identified in existing technology or policy.

It is very common to identify a long list of values. Like the list of direct and indirect stakeholders, students will need to prioritize the values and consider just a small number.

It is also common to encounter tensions among the identified values. For example: (1) Two different stakeholder groups might hold different values; or (2) A designer’s values might be quite different than the explicitly supported project values.

It can be difficult to resolve value tensions. For clarity and transparency, the case studies prompt students to surface value tensions and to propose how they might be resolved.

Method panel 2. Brief example of a value source analysis. (Suggested activity: Sketch a diagram that shows the “web” of values and how stakeholders interrelate.)

Co-Evolution of Technology and Social Structure

Value sensitive design makes a commitment to an interactional stance, which refers to the idea that stakeholders shape the design and use of technology and, in turn, technologies shape stakeholders and society. Accordingly, value sensitive design proposes that both technical and policy elements be considered in technical and engineering design processes.

Each of the case studies, therefore, prompts students to consider both technology and policy elements. Technical elements include system requirements and features, functional capabilities, form factors, physical and logical organization of system components, and so forth. Policy elements include laws, norms and community standards, rules governing allowable and prohibited uses, incentive systems, and forth. Considering both technical and policy elements, expands the design space and positions students to broadly examine the sociotechnical setting.

Method panel 3. Brief example of engaging social and technical elements together when co-evolving technology and social structure. (Suggested activity: Explore the relationships between the technical requirements and the social elements. What implications for design follow?)

Value Scenario

Value scenarios are an envisioning technique for surfacing stakeholder perspectives related to values and technology use. Value scenarios take the form of written hypothetical narratives where stakeholders engage with technologies. Value scenarios are often like a good short story: A protagonist, with motivations and a background, seeks a goal, encounters obstacles, and ultimately gets to some kind of resolution, possibly unsatisfactory. The narrative path generally focuses on the value implications of technology, potential benefits and harms, and unintended consequences. Typically, value scenarios focus on direct and indirect stakeholders, value tensions, and longer-term implications of a hypothesized technology. Value scenarios can be fairly short (50-200 words) or a good deal longer.

In the case studies, students are often prompted to crystalize their technical and policy options in a 200-word value scenario. The following is an example of a relatively detailed value scenario, which has been shortened from the published version and lightly edited (Czeskis, Dermendjieva, Yapit, Borning, Friedman, Gill, & Kohno, 2010):

Method panel 4. Example value scenario. (Suggested activity: Underline the values and direct and indirect stakeholders. How does the narrative clarify what is at stake with this proposed technology?)

Other Methods

In summary, the case studies present design briefs and a suggested design process that prompts students to employ:

• Direct and indirect stakeholder analysis
• Value source analysis
• Co-evolution of technology and social structure
• Value scenarios.

These methods are most often brought together around familiar skills and practices in design thinking, such as:

• Approaches in divergent brainstorming
• Approaches to making choices and documenting assessments with rationale
• Affinity diagramming
• Good use of Post-It Notes and whiteboards
• Summarizing design work with sketches and simple posters.

Notes

For a review of methods in value sensitive design and the foundational studies where they were developed and employed, see Friedman, Hendry and Borning (2017) and Friedman & Hendry (2019).

For more detail on direct and indirect stakeholders, see Friedman, Kahn, Hagman, Severson and Gill (2006).

For more detail on value source analysis, see Borning, Friedman, Davis, and Lin (2005).

For more detail on co-evolution of technology and social structure, see Miller, Friedman, Janicke, and Gill (2007).

For more examples of value scenarios, see Czeskis, Dermendjieva, Yapit, Borning, Friedman, Gill, and Kohno (2010). Values scenarios can also be expressed in video format (see, for example, Woelfer & Hendry, 2009).

For a very thorough discussion of the values and value tensions related to physicians and patients of electronic health records see Grünloh (2018) and Grünloh, Myreteg, Cajander & Rexhepi (2018).