*photos and names are representative to protect privacy
Summary
Context: Science World, a museum in Vancouver, was launching a "makerspace" (a multidisciplinary workshop) for youth to learn STEM skills. Hard skills such as design, coding, and robotics were being taught, but the ultimate goal was to inspire youths to use technology to create their own projects. What was the best way to reach that goal?
Objective: The “maker” community was bringing together exemplars for the K-12 students: adults who constantly taught themselves new tech skills through creative projects. I set out to study maker culture and gather insights that would inform program design in the science museum.
Methods: After an initial period of ethnographic observation, I conducted interviews with a group of makers. A thematic analysis was conducted on the interview transcripts then affinity mapping was used to collect themes into meaningful groups. Once the themes were organized, a survey was designed using the themes as inputs for questions - this was done to test if the findings from the relatively small group of interviewees generalized to a larger population. Surveys were distributed to ~800 respondents through community channels and were analyzed using quantitative methods.
Insights: The themes that generalized most strongly from the interviews fell into three major categories: attitudes towards creativity, problem solving, and independence; skills related to problem solving and finding information; and behaviors that maintained motivation and embedded makers in a community of practice. An additional significant finding was that most makers had a foundation of formal technical training, then built on it with their own independent learning.
Impact: The findings from the research informed the design of the makerspace curriculum at Science World, and during subsequent work at UC Berkeley and SparkFun education. Specifically, they lead to an emphasis on teaching problem solving as a high-level skill, embedding students in a community of practice, and scaffolding training for students so that they could build technical skills before embarking on their own projects.
The research was presented at Stanford's annual edTech conference, led to teacher training courses for dozens of teachers, and informed the design of exhibits, programs, and curricula experienced by over 50,000 learners.
Methods
Ethnographic Observation
I embedded myself into the maker community through a local makerspace called VHS, and through the local Maker Faire event - an annual gathering of the maker community.
The observations from this research informed my selection of interview participants. Specifically, I observed that makers were extremely diverse in their interests - they ranged from artists and crafters to programmers and engineers. This lead me to use a stratified sample for my interview participants. The work also allowed me to identify community channels where I could distribute surveys.
Interviews
Semi-structured interviews were conducted with 13 community members.
The interview group was stratified to capture a comprehensive representation of genders and interests.
Interview questions covered background, practice, community, and attitudes.
Thematic Analysis
I analyzed interview transcripts using thematic analysis. Significant statements were coded with a theme and recurring themes were identified across all participants. This resulted in 32 common themes.
Affinity Diagraming
Themes were gathered onto notecards and an affinity diagraming session was used to group them into categories.
The 32 themes fell into three major categories:
Skills
Attitudes
Behaviors
Survey Design
Interview questions were written to match the themes from the thematic analysis, and were each paired with a 10-point Likert scale of agree-disagree for attitudes and never-very often for behaviors. The goal of the survey was to test if interview themes generalized to a larger sample of respondents.
One issue that was carefully considered was the diversity of subgroups within the maker community. The interviews revealed significant differences between self-identified artists and programmers, so demographic questions were added to the survey along with seven questions about what labels respondents self-identified with.
The questions were piloted with a small group to review wording, then the survey was distributed through several online community channels. Over 800 community members responded.
Insights
Analysis of Survey Results
Quantitative analysis of survey results was conducted, primarily using descriptive statistics. A high average response indicated that the theme related to that question generalized well to the larger sample.
The following key findings emerged:
As a whole, the maker community was well balanced for gender (51% F), but sub-communities were not. Crafter and Artist skewed female, Tinkerer, Engineer, and Hacker skewed male. Maker and Designer were gender balanced - suggesting that these are the most inclusive terms to use.
70% of respondents had some kind of formal training in art, design, or a technical program.
Problem-solving was identified as the most universally valued skill. Attitudes of embracing failure and a strong desire for independence complimented this.
Respondents cultivated habits to maintain motivation such as having multiple projects going at once and having a flexible work schedule.
Respondents also cultivated a community of practice, which they used for maintaining motivation, finding information, and solving problems.
Impact
Insights from this research were used to create the curriculum for Science World's makerspace. Specifically, in addition to STEM skills, the educational staff incorporated the following elements:
"Maker" was confirmed as the most approachable label to use in the space and for the program.
Structured lessons on hard skills were introduced before students were asked to pursue their own creative projects.
The museum secured a booth at the local makerfaire where students could exhibit their work and enrolled students on several popular community websites.
Problem-solving was explicitly taught as an overarching skill (along with an attitude that embraced experimentation and failure as a learning experience)
Makerspace hours were expanded to allow students to have some flexibility in their work schedules, and self-directed work was encouraged during class times.
In addition, the research was presented at Stanford University's edTech conference, led to several teacher training courses, and informed five years of educational design work at Science World, UC Berkeley, and SparkFun electronics.
2
Publications and
Presentations at Stanford University's Education
Conference
Presentations at Stanford University's Education
Conference
5
Years of educational
experience design work in museums and industry
(Science World,
UC Berkeley, SparkFun
Electronics)
experience design work in museums and industry
(Science World,
UC Berkeley, SparkFun
Electronics)
85
Educators trained in
professional development workshops
professional development workshops
60,000
Learners impacted by
products and educational
experiences informed by
the research
products and educational
experiences informed by
the research
"Andrew is a thoughtful instructor, designer, and manager. He creates welcoming user experiences because he understands how to put ideas into practice, both conceptualizing and implementing them in different environments with a range of audiences."
- Sherry Hsi, Research Director, Lawrence Hall of Science, UC Berkeley
Reflection
The mixed-methods approach of interview and survey worked well together. Interviews provided rich data as a starting point for analysis and the survey validated that insights generalized to a much larger sample size.
The diversity of sub-groups within the community was a good example of clustering in a sample of respondents. Although the larger sample was gender balanced, averaging across the entire sample would have hidden some important gender effects related to sub-groups.
As the research was presented outside of the initial Science World conference it was interesting to see how different groups (educators, educational researchers, design researchers) had different orientations towards research. This underscored the importance of considering the final audience when planning and conducting research.