Illustration by Shyama Golden (published on Slate.com)
“T-t-tell me what you did to yourself … and why?” She shut her eyes for a moment and took a deep breath. Just pushed out baby, she told herself. Pain is just from that. I’m OK. I’m OK.
“I’ve listened to you,” Obi 3 said. “One day, you said you wished someone would protect you like you protected the baby.” […]
“Then you decided to find a way to protect me.”
“Yes. I invented a way, then I built my invention.”
“Necessity is the mother of invention,” Anwuli said, with a weak smile. “Wow. Technology harbors a personal god; my Chi is a smart home.”
PROMPT
How can speculative futures and imagined technologies allow us to raise questions about Cyber-Physical Systems (CPSs)? How can we expand upon such works to interrogate interfaces, assurance, and agency? What are the cybernetic engineering techniques and tools that can help us uncover and unpack CPSs and their implications for a safe, responsible and sustainable future?
SETTING THE SCENE
As can be explored in more detail in this post HERE, the role of speculative fiction in enabling future practitioners of a New Branch of Engineering (NBE) to design, construct, commission, manage and decommission CPSs is a topic that I have found a particular interest in over the past year. Speculative Fiction/Futures (SF) is an overarching genre that captures works whose narratives explore the long-term implications of current realities (often technological) as a means of reflecting on today’s anxieties and yearning. By extending the signals that emanate from various domains, SF allows us to reflect about humanity's place in the future, and the potential role of technologies in the times ahead. Because the imagined worlds that come out of such works are often based on science and current trends, the methods behind “world building” (Johnson, 2011) provide meaningful and established ways of imagining and interrogating future uncertainties in complex systems. It comes as no surprise then that decisions makers from a wide range of sectors have found it useful to apply such methods of forecasting and backcasting in quest of short-term and long-term goals alignments. If thinking about possible and probable futures can help uncover generally unintuitive scenarios that inform intent and guide decisions along the process of creating technologies, how can this method be enhanced to form new ways of understanding the development of interfaces, and our expectations of assurance and agency with regards to future CPSs?
This image was adapted by Joseph Voros from the work of Hancock, T. & Bezold, C. (1994). Possible futures, preferable futures, Healthcare Forum Journal, Vol. 37, No. 2, 23-29.
Mother of Invention: A Systems Analysis seeks to explore the concept of agency, assurance and interfaces within increasingly complex systems by looking at a particular cyber-physical system described in Nnedi Okorafor's story. In Okorafor's "Mother of Invention" (you can read the full story HERE) we are transported to New Delta City, Nigeria (present day Niger Delta,Nigeria) where we are invited to follow Anwuli, a soon to be mother, and her smart home Obi3, as they make their way out of an environmental disaster. This world, set 100 years in the future, is a depiction of complex systems and the unforeseen results of our interactions with our environment and technology.
So how can we go about understanding the limitations of the CPS imagined by Okorafor and its implications for human agency, and assurance of safety?
The debate around agency as it relates to cyber-physical systems is often tied in with the idea that to have agency is to also have autonomy. That is, the ability to dictate or determine goals. As a result, an almost binary categorization of human vs technology occurs where agency and autonomy are expected to remain in tandem. But cultures/theories that understand technology to be an extension of the human begins to blend this view. This is a point raised by Jenny Hall’s study of Japanese artisans, and their deeply mutual relationship with their tools as not just valuable objects, but spiritual extensions of themselves. Here, we begin to see that the agency afforded to the tools emanates from the expressed autonomous goals of the artisans. The tools are described as being no different than the physical bodies that humans act with- goals seeping from mind to body to “body.” Obi3 is no different, though perhaps an even more capable agent. As it developed the protective womb that saved Anwuli and her newborn’s lives, it was only acting on behalf of Anwuli’s wish (though unintentional) to protect her like she does her then unborn daughter. Obi3’s actions are the expressions of Anwuli’s request. A more interesting question then perhaps, is whether Obi3’s actions are extensions beyond this granted agency. In other words, what are the implications of Obi3’s agency within the feedback loop of these human-machine interactions?
By first mapping the overarching elements and interrelationships of this imagined world using a causal loop diagram (CLD), and then dissecting Obi3, the smart home of the future, through a System-Theoretic Process Analysis (STPA) I hope to shed light on the potential application of a combinative systems/behavior mapping and STPA method, situated within speculative fiction, as a critical assessment tool for the development life-cycle of future CPSs.
RESEARCH METHODS
APPLIED METHODS
Unpacking systems
“a system must consist of three kinds of things: elements, interconnections, and a function or purpose.”
Donella Meadows- Thinking in systems
To explore Okorafor's work, we must first establish why it is useful to view her imagined world as a system. Though the particular considerations of safety and assurance will eventually require us to remain within the bounds of Obi3 as the CPS of interest, to appreciate the social, environmental economical, and political system within which this technology sits requires that we develop an appreciation for the many elements and interrelationships that compose this world. While all models of systems are inherently simplified versions of complex relationships that are captured within the bounds of the intended analysis, visualizing the different interactions between the different agents described in Mother of Invention can be a meaningful tool in understanding the relational implications of the CPS that we seek to interrogate here. In some ways, applying this tool in the context of existing speculative fiction provides an easier way in for the practitioner of an NBE, because unlike a forward facing endeavor, the creator of such works are likely to have already wrangled with the critical elements of the imagined system, and as such offer a strong foundation for the application of a systems lens. Of course, this shouldn't limit the practitioners ability to modify the boundaries established by the authors of such works. In fact, as we will see in the next section, as we construct causal loop diagrams to visualize the influence of components within the system, we are forced to aggregate or disaggregate some of the elements captured by Okorafor.
“In order to tackle a problem situation we need to understand the basic behaviour (dynamics) of the system of interest. How does the state of the system change over time in response to internal and external forces? To talk about how a system changes over time, we must consider changes in essential properties or aspects of the system."
Katrina Proust & Barry Newell- Constructing influence diagrams and causal loop diagrams
Mapping System Behavior: Causal loop diagrams
Definition
Causal loop diagram: a causal diagram that aids in visualizing how different variables in a system are interrelated. The diagram consists of a set of nodes and edges. Nodes represent the variables and edges are the links that represent a connection or a relation between the two variables
When thinking about CLDs, I found that applying Proust and Newell's practical step-by-step approach that facilitates the conception and construction of Influence Diagrams (ID) and Causal Loop Diagrams (CLD) to be quite useful. While I had initially approached this procedure informally, relying on my understanding of CLDs that came out of classroom sessions and homeworks, I realized that following Proust and Newell's methodical process unearthed some faulty identifications (wrongly named variables for example) and allowed me to also reflect on some assumptions and decisions I had taken along the way, highlighting any simplification or complication I brought to the visualization of this system. This gave me an appreciation for establishing not only accessible but also transparent processes that can allow future practitioners the ability to deconstruct the different phases of a particular output.
Informal attempt at creating a causal loop diagram of the world imagined in "Mother of Invention"
Using the same variables I identified in the informal process visualized above, I began to filter them through the step-by-step process as follows.
Step 1- Variables
As mentioned earlier, the task of identifying variables, at least as a starting point, has been facilitated by the author's own work of creating this speculative world. In a separate application of this step, we would be asked to identify the relevant issue at hand or the problem that we are concerned with. Here, we assume that the author has captured the elements that have some important role in the system. This also reduces the difficult task of setting a boundary since a complex system is impossible to capture in its entirety. As such, we begin this process with a thorough reading of the short story, and writing down the elements and keeping note of the interactions that exist between them.
Elements/entities represented in "Mother of Invention":
Anwuli, Obi3, Nigeria, Igbo, Festus Nnaemeka (neighbors), grocery/supply shop, husband/wife, drained swamplands, Dr. Iwuchukwu, new delta, friends/family, daughter, swamplands, riverways, oil, oil corporations, conflict, world famous green grass, grass plant (bread, liquor, plastic, and so much more) livestock animals, erosion, New Delta city, smart cars, electric scooters, peri flower/periwinkle grass (hybrid of sunflowers, zoysia, rice, jasmine), Chinese labs, Nigerian scientist Nneka Mgbaramuko, periwinkle seed (as food), harvester trucks, climate change/climate chaos, pollination, pollination misalignment, wind ( increases pollination), allergies + Izeuzere, New Delta citizens, pollen tsunami, migration, anti-allergen injections, arid environment safest, ambulance, Chinese corporations, Nigerian government, deaths, population, local supermarket.
Step 2- Naming variables
When revisiting my first attempt at a CLD I realized this step to be the most critical missing part because I had not captured the capacity for change that should be reflected in the identified variables. For example, I listed Anwuli, the protagonist of the story as simply "Anwuli", ignoring the fact that I had buried my understanding that this variable would represent Anwuli's comfort level. As Proust and Newell add, a useful way of naming variables is to include descriptive terms such as "amount, number, extent, area, size, level, degree of, etc." Applying this step allowed me to recapture the variables identified in step 1.
Anwuli's level of comfort, Obi3's abilities, Nigeria's living conditions, Igbo population, Festus Nnaemeka (neighbors' level of support for Anwuli), number of grocery/supply shop, husband/wife's level of support for Anwuli, size of drained swamplands, Dr. Iwuchukwu degree of effectiveness, livability of new delta, friends/family's level of support for Anwuli, daughter's comfort level, number of riverways, amount of oil, number of oil corporations, level of conflict, popularity of green grass, number of uses of grass plant (bread, liquor, plastic, and so much more), amount of livestock animals, erosion level, number of smart cars, number of electric scooters, amount of peri flower/periwinkle grass (hybrid of sunflowers, zoysia, rice, jasmine), amount of research in Chinese labs, reputation of Nigerian scientist Nneka Mgbaramuko, amount of periwinkle seed (as food), number of harvester trucks, level of climate change/climate chaos, level of pollination, frequency of pollination misalignment, amount of wind ( increases pollination), number of allergies + Izeuzere, number of New Delta citizens, frequency of pollen tsunami, extent of migration, frequency of anti-allergen injections, size of arid environment, number of ambulances, profitability of Chinese corporations, profitability of Nigerian government, number of deaths, size of population, access to local supermarket.
Step 3- Selecting variables + Step 4- Clarity of Expression
The next step invites us to provide more specific units of measurement for the variables we have identified and have decided to keep in the diagram. Though specified in the next step 4, I found it more useful to begin aggregating or disaggregating variables at this stage. By picking the variables that are of interest to us, we can both save time (avoid creating units of measurement before aggregating/disaggregating) and begin to create a clearer understanding of which variables we are identifying as central to the provocation we want to answer. In this case, we're already noting the importance of Periwingkle grass in the dynamics of the system we are trying to understand. Once we identify the final list of variables, we can return to step 3 and assign units of measurement. Another error that this step-by-step process uncovered is also the fact that I had included a leading term in level of climate "chaos", whereas a more neutral definition of the term/variable would be level of climate change. These changes are reflected in the table below- which is also a good way of organizing the variables before we begin to illustrate them as a diagram.
Step 5- Links and Loops + Step 6- Constructing an Influence Diagram
Finally, using the variables identified in the previous steps I revisit the diagram creation process. Using the LOOPY tool once more I am able to create the diagram which reflects the interconnections through the arrows, and include the polarities which transform the Influence Diagram into a Causal Loop Diagram. As a result of the more thorough approach provided by Proust and Newell, I'm able to clean up and correct a few interactions I had misidentified or polarities that I improperly captured. I was also able to apply the LOOPY tool itself more precisely, by setting the start of the model simulation to what I considered to be appropriate starting amounts (as opposed to my first use which incorrectly used this feature to differentiate between elements.) A playback of the simulation below shows more insightful (and correct) connections compared to the informal approach.
causal loop diagram of the world imagined in "Mother of Invention
STAMP & STPA
“Lessons learned over centuries about designing to prevent accidents may be lost or become ineffective when older technologies are replaced with new ones. New technology introduces unknowns into our systems and even unk-unks(unknown unknowns).”
Nancy Leveson- A New Accident Model for Engineering Safer Systems
The STAMP Model ( Systems-Theoretic Accident Model and Processes )- is a safety/risks assessment model that defines accidents as things that are caused by inadequate control, and thus tries to identify how these issues can be addressed early on in the design process . One of these processes is the STPA or the Systems-Theoretic Process Analysis, which is what I'll be using to analyze the design of Obi3.
Here, we must note that, despite having looked at the larger system, we are instead choosing to hone in on the Operating process that involves Anwuli and Obi3. This analysis could have easily been applied to the different components of the larger system operations as seen in the figures below.
The Operating Process is seen in the bottom right of the first figure, we then open up this Process to the STPA analysis through a step by step process.
Step 1
Definitions
Accident – An undesired or unplanned event that results in a loss, including loss of human life or human injury, property damage, environmental pollution, mission loss, etc. - May involve environmental factors outside our control.
Hazard - A system state or set of conditions that, together with a particular set of worst-case environment conditions, will lead to an accident (loss). - Something we can control in the design.
Identify accidents and hazardsObi3 misunderstands an instruction (hazard) and executes unsafe maneuver that results in damage/loss of life or property damage (accident)
Obi3 is unable to monitor pollen tsunami conditions (hazard) and cannot enter safety mode resulting in Anwuli and her daughter’s death (accident)
Obi3 is in a state of elevation (hazard) when the occupant (Anwuli) steps out of the house leading to physical injury or the loss of her life
Identify system safety constraints
Identify system safety constraints that can be put in place for preventing/minimising the likelihood of the potential accidents and hazards that have been identified.
Obi3 misunderstands an instruction (hazard) and executes an order that results in damage/loss of life or property damage
Safety constraint: Obi3 should confirm owner’s instructions before executing order
Obi3 is unable to monitor pollen tsunami conditions (hazard) and cannot enter safety mode resulting in Anwuli and her daughter’s death (accident)Safety constraint: The operator (Anwuli) should have a secondary monitoring mechanism for pollen tsunami alerts (phone/radio)
Obi3 is in a state of elevation (hazard) when the occupant (Anwuli) steps out of the house leading to physical injury or the loss of her life
Safety constraint: Obi3 should not be able to open any doors if it has any elevation
Draw the control structure
The control structure identifies the components in the system and captures the operation of those components.
By closely examining the reading, we identify the following components and functions of Obi3: Speakers, drones, mechanized cushioning beams, rotation capabilities (follows sun), transform shape, repair/build mechanism, holographic display, assistant interface, cooking, lights, music (speakers), sensory lights (electromagnetic noise), phone/text, backup solar generator, ceiling display, Natural Language Processing, medication, utilities, duct to filter pollen, phone’s grocery list, research(input) = news, central person, environment, home’s filter (nosy and intrusive otherwise), formal logic, protective egg, walking
Because some of these components showed redundancy (music through the speakers for example) I eventually combined them as seen below. I also decided to capture the rooms in the home as opposed to focusing the components that could be duplicated in them.
Identify unsafe control actions
A control action is an action put in place for using the system.
We then identify some unsafe control actions within the smart home system.
Step 2
Determine how these unsafe control actions could occur
Do not move command: Anwuli/the operator could choose not to move from their current location, forcing Obi3 to remain in place despite the potential for the pollen tsunami breach of its air filter system, leading to potential human loss.
Turning off monitoring “filter”: The occupant could choose to turn off the monitoring/privacy filter to allow Obi3 to get as much information as possible from its central person and enable more personalized assistance
Feedback/message interruption: The occupant could choose to ignore a message (some sort of fatigue) and disrupt Obi3 from delivering a critical message
Identify causal scenarios
Casual scenarios refer to potential errors or faults that can occur at any point when using a system. Examples of causal scenarios include: Component failures, Inadequate operation, Incorrect inputs, Missing information, Delayed operation
We then identify some of these causal scenarios.
Beams get stuck: Obi3’s cushion beams(turned legs) could stop working properly because of a mechanical issue, disabling its mobility/elevation functions
Obi3 receives false information: Obi3 could receive the wrong planning information from one of its sources which is news. If the latter comes is retrieved from misinformed/disinformed sources it could lead to faulty decision making.
CONCLUSION
Taking on "speculative determinism"
I thoroughly enjoyed looking at this fictional world of the future. As I was exploring this environment I reflected about the opportunities that such imagined worlds and such technologies could open up in allowing us to interrogate the technologies of today, their precursors.
Friedman and Kahn’s research is an interrogation of computer system design (CPSs for our purposes) and its recursive impact on human agency itself. For them, Obi3 represents a trifecta of computer practices that are problematic for human agency. Their first concern, that of “anthropomorphizing computers”/granting them intentionality, has been addressed here in the previous paragraph to the extent that CPSs should be understood as extensions of the human, and not a duplicate entity. Where Obi3 begins to become problematic is with regards to the closed loop it develops as it sets intermediate goals in pursuit of its “assigned” task. Obi3 delivers on Anwuli’s intent as a present or surprise, arguably hiding its process along the way (Anwuli enabled its materials gathering to an extent, but the unawareness stands.) From studying medical complications, to crafting a ventilating device, Obi3 was allowed to “delegate its own instructions”, maybe even directing Anwuli herself as an agent at times within this loop. This challenges Anwuli’s autonomy, as a sort of loss of temporal control of her intentionality. Would she have chosen to act differently if she was aware that Obi3 was pursuing the directive for so long?
Technologies continue to serve our intentionality. Obi3, an example of a future CPS, demonstrates the heights of the agency we grant our tools. But we must revisit this agency we so comfortably endow them. We need careful considerations about the interfaces that we are creating, with an eye towards preserving humanity’s autonomy, and our ability to change our minds and reimagine our goals as we express ourselves within this feedback loop. Obi3 is an extension of today’s conception of a smart home, and if we can extend the signals or trends that we have observed within the present application of technologies like Alexa or Google Home-- the issues of privacy, natural resource extraction, labor conditions, and so on -- then perhaps it’s possible for us to interrogate the design, construct, commission, manage, and decommission of Cyber Physical Systems, or what Jasanoff et. Al. call "sociotechnical imaginaries", in their own playground.
REFLECTIONS
Where from and where to
I was grateful to have undertaken this exploration, though frustratingly I had to rediscover my words when I lost my first iteration. So perhaps that is the first lesson, that of time management and always backing up your work. One of my goals here, besides applying systems analysis to speculative fiction as a way into futuring methodologies, was to get a better grasp of systems mapping by practicing my application of causal loop diagrams. This wasn’t without its limitations as briefly mentioned above. I still struggle to identify reinforcing or balancing loops within the interrelationships presented in the short story. I could also perhaps look into more descriptive methods like stocks and flow diagrams to provide the reader an even more pointed understanding of the system we’re exploring. Additionally, I realize that the STPA analysis provided here is somewhat limited in that it only identifies some considerations within what’s usually a much more detailed analysis of safety risks, constraints, and possibilities for mitigation. These are both methods that I would like to develop and internalize as the semester progresses. By reapplying a more methodical way of creating causal loop diagrams, I believe I was able to shift the needle a little further in the right direction. Finally, as someone who is seriously interested in the potential of speculative fiction in guiding our future, I’d like to continue learning more about speculative fiction methodologies and how they inform the development of these works. In other words, I’d like to get a handle on how authors navigate the realization of such works from idea to fictional reality. For now, I am happy to have applied some methodological rigor to uncover the present realities presented to us in such visions.
Obi3- Before it transformed itself to save Anwuli. As imagined by me
Skills Gained
Systems dynamics/ systems mapping by visualizing elements through causal loop diagrams
The extent of my knowledge of systems dynamics/mapping was limited to skills gained in the classroom and through HW. Having not performed well on this particular assessment, I wanted the opportunity to get a better grasp of this process. I was under the impression that my informal application of the causal loop diagrams within the portfolio was sufficient (and in fact correct) by mid-semester. But leaning on feedback from the Build team, I revisited some of the resources associated with this method. This was extremely helpful and I feel as though I can easily reproduce these results, and perhaps more confidently so this time around. I believe that I have reached an intermediate level when it comes to Influence Diagrams/Causal Loop Diagrams.
Safety/risks assessment outline through Systems-Theoretical Process Analysis within the STAMP framework
STPA is a now established method for system level safety analysis, and so I wanted to apply it in the speculative fiction context in order to bring the rigor offered by the analysis to futuring methods. Because STPA itself is a very detailed and demanding process, I was never under the impression that I would produce what amounts to a full application of the analysis. Nonetheless, I wanted to internalize the knowledge picked up during the simulation and STPA skills HW in order to have a working grasp of the method. I consider my theoretical understanding of the method to be just under intermediate, while I can apply a basic form of the method to quickly glance at system safety.
Imagining future states of a CPS through Speculative Fiction
Having spent a good amount of the semester exploring concepts of world building, scenario planning, speculative design, applied the method across the CPS Analysis and having attended virtual futuring sessions, I believe this is the skill that I have developed the most over this term. I considered myself to be a bystander or consumer of such works in the past, and I am excited to have learned the techniques to begin developing works in the same space. I believe I have an intermediate to advanced skill level in this category.
Applying vectors of the 3A Institute by exploring Agency, Assurance, and Interfaces
My understanding of the vectors is somewhat still malleable. Still, I believe that the methods used within this portfolio have led me to consider more insightful questions about these vectors, and the relationships between them. Because I spent most of the semester within the exploration of the "Intent" vector, I would like to revisit the works of my cohort members and the Institute's continued research to understand these components of the framework a bit more. I have developed a basic to intermediate level of skill with regards to the vectors.
References
Donella Meadows (2015) Thinking in systems: A primer
Friedman, Batya and Kahn, Peter (1992) ‘Human Agency and Responsible Computing: Implications for Computer System Design’
Hall, Jenny (2015) ‘The Spirit in the Machine: Mutual Affinities Between Humans and Machines in Japanese Textiles’
Hancock, T. & Bezold, C. (1994). Possible futures, preferable futures, Healthcare Forum Journal, Vol. 37, No. 2, 23-29.
K. Proust, B. Newell (2020) Constructing Influence Diagrams and Causal Loop Diagrams
Nancy Leveson (2011) Engineering a Better World