Step 6

6.1 Questions

Making people aware and getting them involved to the point where they want to participate is difficult so you have to show quantitative and qualitative results in the right way to keep the public engaged. At this point there are also operational, tactical and strategic considerations that relate to accountability issues and follow-up. For example re-establishing ownership, governance, escalation paths, roles, and responsibilities among other considerations such as:

• Can you continue by designing interventions and involving the public?
• Are you measuring changes, results, and social impact?
• Are you being transparent and publishing the data and results with visualisations and engaging stories?
• Have you been speaking with the public to generate a “data plus dialogue”?
• Are you communicating your project, creating awareness and accountability?
• How can you reflect on the impact you have made in the context of your project goals as well as in an ethical context taking into account social values?
• Is it possible to keep momentum and enable a continuous discussion?
• Who will now own this project and how will they be accountable?
• Have you established a follow-up plan and strategy?

6.2 Project Retrospective

Now is a good time to reflect on your experience, look at the original intent, review your goals and revisit the Decision Canvas you worked on in the beginning of your journey to responsible and ethical sensing. Project post-mortems and assessing lessons learned is important but think about the unique aspects of sensing and your project in a broader sense. It’s not just about measuring performance against KPIs, it’s also about how you did or did not achieve your goals in responsible and ethical ways. Here are some suggestions to help you assess your project’s impact:

• Revisit your original Decision Canvas with project stakeholders.
• Think about Lessons Learned. Document and share your experience.
• Measure the changes you have made with an impact analysis report to quantity the before and after of your project and any interventions you conducted.
• Publish the results and tell your story. It is important to be transparent and share your results.

No doubt your organisation will have its own way of reflecting on a project. We hope to add some value at this stage by emphasising the ethical aspects specifically related to sensing projects. Now would be a good time to (re) visit the Responsible Sensing Lab and read about the projects: Human Scan Car, Transparent Charging Station, and Shuttercam. Three well documented projects you can refer to.

6.2.1 Reflecting: Social and Ethical Impact

It is difficult to measure success in a sensing project you may or may not have achieved the goals you established. You may have had some serious problems or surprising results but what is most important is understanding how your project impacted the public. Did you change perceptions or behaviour and what does this mean? After evaluating the quantitative results of your project you should consider engaging the public again to get some feedback and gather the qualitative data about social impact.

6.2.2 Evaluating Ethical Performance

Responsible and ethical sensing as a practice is just now beginning to take shape. If we design and implement our projects keeping social values and ethical principles in mind we are pioneering important work. The impact we create in our cities is much more than solving problems that have to do with how people move in public spaces but how we approach solving these problems with privacy and dignity at the same time.

Now is a good time to hold an Ethical Performance Workshop with your project stakeholders. You can revisit the Applying an Ethical Framework and Measuring Ethical Principles in Step 3 or visit some of the data ethics links below to help you think about how to evaluate your ethical performance:

• Privacy by Design
• Tada! Data Disclosed
• Responsible Sensing Lab
• Cities for Digital Rights
• Value sensitive design
• Utrecht Data School

Project stakeholders should be evaluating the ethical performance for each of their experiments. Teams are free to define their own values and principles, but most importantly see how they can continually improve on responsible sensing during the project life cycle. In the illustration below, you see a comparison of how teams ranked themselves on a hypothetical research project (based on the Tada ethical framework).

6.2.3 Follow-up

At this point there are also operational, tactical and strategic considerations that relate to accountability issues and follow-up. For example re-establishing ownership, governance, escalation paths, roles, and responsibilities among other considerations.

In agile projects there are various approaches to continuous improvement. Specific to responsible and ethical sensing it is useful to map out your journey, consider what impact you have had, and determine a path forward or a productive conclusion. Here are some considerations:

• Can you amplify the positive impact you have had with your project?
• Is it possible to scale, duplicate, and/or distribute your project?
• If you continue, how can you improve your project responsibly and ethically?
• If you conclude your project, how can you and the public benefit from lessons learned?

When we talk about agile development, design thinking, or experimentation in Living Labs it’s important to remember that these are all non-linear approaches to creative collaboration and conclusions are less important than applying lessons learned to improving your project or designing a better solution to test again. Here are some links to help you think about follow up so you can build on your experience and continue to create impact with your project.

• Agile software development
• 5 Stages in the Design Thinking Process
• Experimentation
• Embedding experimentation in government
• Urban Living Labs: A Living Lab Way of Working