Keywords

1 Introduction

In today’s connected world, several projects are developed asynchronously and in distinct geographic parts. Open-source initiatives for sharing source code and documentation play a significant role in project development [7, 18]. Package managers, repositories, virtual environments, and containers such as npmFootnote 1, PyPIFootnote 2, GitHubFootnote 3, and DockerFootnote 4 accelerate software development and allow developers from all around the world to run pieces of software seamlessly with few configuration requirements and, consequently, easily collaborate. Parametric design software empowers designers, developers, and end-users to adapt 3D models to their contexts and needs. Digital fabrication techniques such as laser cutting and 3D printing enable physical realization through digitally shared models. Additionally, there is an abundance of affordable and fast alternatives to produce printed circuit boards. This conjecture makes it possible to not only share pieces of code but also distribute complete instructions for replicating a particular physical artifact with a fine-grain level of detail.

All the mentioned collaborative tools can be used to aid in Digital Musical Instrument (DMI) design. DMI is a class of physical interactive objects that comprise a gestural control unit, mapping strategy, and sound synthesis module [11]. The development process of DMIs presents an exciting field of study as it covers different areas of skills that demand diverse approaches to share code, diagrams, schematics, 3D models, and document the replication process as a whole.

A number of aspects need to be considered for DMI design [3]:

  • Mechanical structure (shape, color, appearance),

  • Electronics (circuits, sensors, power unit, microcontrollers, actuators, communication modules, storage modules),

  • Programming (firmware, software, communication protocol),

  • Mapping (connections, data processing, data modification), and

  • Sound design (synthesizers, DAWs, VSTs, samplers, drum machines).

A structural view of these aspects can be seen in Fig. 1. When the instrument design is a collaborative project, information regarding those aspects must be accessible to all participants, hence the necessity to adapt and use collaborative tools in DMI design.

Fig. 1.
figure 1

Different Skills in DMI Development Process

Full size image

2 Motivation

While there are many researchers, startups, and makers experimenting and developing novel musical instruments, there is often little communication between these innovators.

Although there are few examples, the reach of scientific publications on DMIs does not usually extrapolate academic audience. Additionally, startup companies are market-driven entities. In other words, they are limited to what sells well and interested in products that are compelling to a particular market. Considering the maker community, the driving force for individuals is based on what satisfies her or his needs. Thus their focus is specific. In sum, the paradigms are quite different from each community [10], but through an open innovation perspective, these communities can share knowledge and experience.

Fig. 2.
figure 2

Integration of DMI’s innovators

Full size image

We believe that, by sharing projects, these three innovators (Fig. 2) can collaborate, distribute technology, code, design files, and learn from each other.

3 Design Process and Replication

The DMI design process is reportedly idiosyncratic in literature [1]. Usually, the instrument is conceived and developed for one performer, or a person plays the roles of performer and designer. The process often varies from the user’s context and intention, and, typically, there is little or no structured thinking about the process. We could speculate that can be the reason why many digital musical instruments are not adequately documented.

In an attempt to formalize the mindset, we borrow definitions from design theory for DMI context [5]. So we could formulate that the design process of a digital musical instrument is a set of steps that the design team, along with users, follow to achieve the desired result. The desired result is not an unchangeable monolith, but it evolves during the exploration and experimentation throughout the project. The process is, therefore, often cyclic and incremental. Features can always be improved from one cycle to another.

Inspired by processes from different areas such as engineering design, mechanical design, creative thinking, user-centered design, and innovation, we present a description of a general design process [2] is presented in Fig. 3. In the Definition phase, the team typically works on understanding the concepts behind the artifact, such as stakeholders, scenarios, shared knowledge of the area, the user’s intention, and context of use. It defines the restrictions that comprise the project’s design space. In Idea Exploration, the team explores possible paths in the design space, generating and selecting ideas that conform to the user’s intentions and contexts of use. During Prototyping, the team realizes the abstract ideas into an artifact that can be utilized and tested, with which the user can interact. Finally, in the Evaluation, the team validates whether the artifact is adequate to the user’s intention and context of use, rising corrections, enhancements, and modifications for the next cycles.

After finishing one or more cycles, the design team has a set of outcomes from the process typically. For instance, code, diagrams, schematics, 3D models, in sum, assets used to represent ideas. Working prototypes, general notes, and audiovisual records are also typical outputs from some cycles of the design process.

Although the process is cyclic and incremental, there is a moment that the implemented features partially fulfill the essential user’s requirements. This is the moment in which the design team defines the release candidate that works as a project snapshot, i.e., the current status is satisfactory; therefore, one should save it. In Software Development, the semantic versioningFootnote 5 is commonly used to define major, minor, and fixing versions of the system.

Replication is the process of reproducing the project outside the environment in which it was developed. The replication documentation consolidates the project assets, working prototypes, general notes, and audiovisual records in a way to facilitate the comprehension of the project by an outside maker. After the development, the replication documentation is a reflection of the process and all the steps that were taken until the current version of the system. Besides, it is an empathy exercise, since the design team must put themselves into the maker’s position, considering their possible lack of specific skills, mistakes, misinterpretations. It is a process of communication that can be arduous and more difficult than only implementing the system.

The design team should not only think about the maker and the replication process but also consider the operating instructions, which should also be explained. To raise the complexity, besides presenting a possible way of replicating a project, the design of the project can allow for individual modifications. With the parametric design, the designer does not define the final object but a set of rules. The users, in their turn, manipulate parameters that, along with the rules, produce the final artifact.

In sum, the replication documentation is a deep dive into the abstract parts of the developed project and constitutes an essential step for making projects accessible to makers and users. In this paper, we intend to reflect on the documentation process for the replication of a digital musical instrument, discuss possible challenges and present a checklist to help designers and developers on the arduous process of documenting for replication.

Fig. 3.
figure 3

Design process and replication documentation

Full size image

4 Objective and Proposal

The main objective of this paper is to aid digital musical instrument designers to share their projects by providing ways that other people can reproduce them with the least effort in the shortest time possible.

This paper discusses approaches to the process of online sharing and documenting collaborative open-source projects of digital musical instruments. We examine the development process of DMIs and how can we expose the steps of this process into the structure of the source code repositories and how can we standardize the documentation to ease the comprehension and communication with collaborators.

We based our proposal on: 1) the previous experiences that our laboratories and research groups had during the development of different DMIs: the T-Stick [8, 13], the Giromin [17], the Controllerzinho [3], the Pandivá [1], TumTá and Pisada [16]; and 2) DMI related toolkits (Probatio [2, 4], Prynth [6]). The collaboration and attempted DMIs reproductions are the main motivation to develop this project.

As a contribution, we expect to offer initial guidelines and recommendations for DMI designers to share their projects, focusing on replication.

Inspired by the open-source hardware certification, we propose a checklist for sharing online projects in the form of questions. The checklist follows the different areas related to the musical instrument development process.

5 Inspirations

Although there are several examples in software development, the DMI development process presents a few structured attempts to organize how a project is shared and how the documentation covers the replication or reproduction of the artifact [6, 12, 14, 15, 19]. Due to the different sets of skills and their specific related tools, sharing and documentation is a challenge. The majority of the New Interfaces for Musical Expression (NIME) papers, for example, are concerned with a functional description, but only a few presents the means to reproduce the artifact. Consequently, this situation hinders communication between the project’s collaborators or makers interested in replicating the instrument.

5.1 Open Source Hardware Certification

Regarding the lack of a standard definition of what an Open Source Hardware (OSHW) is, there have been many initiatives to create a certification program related to specific definitions. The Open Source Hardware Association (OSHWA)Footnote 6 holds a community ran a model of advocates for OSHW which helps the community to build a standard definition and stimulates designers to understand what they could and should do to release their projects as OSHW.

As stated in the association’s website: “Open Source Hardware (OSHW) is a term for tangible artifacts – machines, devices, or other physical things – whose design has been released to the public in such a way that anyone can make, modify, distribute, and use those things. This definition is intended to help provide guidelines for the development and evaluation of licenses for Open Source Hardware.”

The certification process is free and gives the product a unique ID for each OSHW. The benefit of the certification is that it helps the maker community to discern projects that are open from those that are only advertised as opened but have incomplete documentation. For instance, when the designer shares only a few low-resolution images of the PCB schematics instead of the actual CAD design files or the GERBER files.

Below, we present the guidelines organized by OSHWA to help the developers organize their projects to the certification process.

  • Documentation

    • Have you made your original design files publicly available?

    • Have you made any auxiliary design files publicly available? (optional)

    • Have you made a bill of materials publicly available?

    • Have you made photos of your product at various stages of assembly publicly available? (optional)

    • Have you made any instructions or other explanations publicly available? (optional)

    • Have you properly licensed your design files so that others may reproduce or build upon them?

  • Hardware

    • Have you provided links to your original design files for your hardware on the product itself or its documentation?

    • Have you made it easy to find your original design files from the website for a product?

    • Have you clearly indicated which parts of a product are open-source (and which are not)?

    • Have you applied an open-source license to your hardware?

  • Software

    • Have you made your project’s software publicly available?

    • Have you applied an open-source license to your software?

They also suggest good practices after the certification process like labeling the hardware with a version number or release date, so people can match the physical object with the corresponding version of its design files and to use the OSHWA certification mark logo.

All the so far 395 licensed OSHW projects can be easily found in their websiteFootnote 7, helping makers to find projects they can make or modify as well as giving a big set of examples of ways to structure a new OSHW project for the community.

5.2 Other Initiatives

Maker Community: The Maker community is a substantial inspiration to our effort with motivating initiatives such as Instructables’Footnote 8, Sparkfun’sFootnote 9, Adafruit’sFootnote 10 tutorials; Thingisverse’sFootnote 11 online 3D printing models sharing, and Autodesk Fusion 360’sFootnote 12 online parametric design editor to name a few. Maker communities usually aim to provide images, videos, schematics, organized source code, and well-prepared content. There is also a focus on encapsulating technical details for beginners: Adafruit and Sparkfun have their libraries for Arduino, for example.

NIMEhub Initiative: The NIMEhub workshop [9] presented in 2016 during the International Conference on New Interfaces for Musical Expression reinforces the importance of the discussion about the replication process of Digital Musical Instruments in the context of the NIME community. Unfortunately, the results of the workshop discussion were not published. It was mentioned the use of standard approaches to share DMI designs, and we could see little advances towards the main subject of the workshop in the context of the NIME community.

6 Case Studies

In this section, we present projects developed by our research groups and laboratories, with which we could reflect upon the replication documentation process.

Fig. 4.
figure 4

T-Stick sopraninos (http://www.idmil.org/project/the-t-stick/).

Full size image

6.1 T-Stick

The T-Stick is a family of DMIs designed and developed since 2006 by Joseph Malloch and D. Andrew Stewart at the Input Devices and Music Interaction Laboratory (IDMIL) at McGill University. Expert performers and composers have used the instrument as part of their musical practice, including D. Andrew Stewart and Fernando Rocha. It has been used in dozens of public appearances in countries such as Canada, the USA, Portugal, Brazil, Norway, and Italy.

From 2006 to 2018, there were no major hardware or software upgrades or modifications in the T-Stick. In 2018, Alex Nieva performed a significant upgrade [13], replacing hardware components for more modern versions, more noticeably the replacement of the accelerometer for a complete Inertial Measurement Unit (IMU), containing accelerometer, gyroscope, and magnetometer. The software was also revised to run on more recent microcontrollers, although the most significant modification was the ability to work wirelessly.

All documentation regarding the T-Stick development before Alex Nieva’s research, as well as building and usage instructions, could only be found at academic publications and through IDMIL’s archives. In October 2018, during the upgrading process, Nieva created a public repositoryFootnote 13 to organize documentation and make it publicly available. The documentation included the first version of the building instructions for the T-Stick Sopranino Wi-Fi, and it was used by McGill students in 2018 to build four instruments, as an assignment for a Music Technology course taught by Marcelo Wanderley in the same year.part o.

The building assignment was repeated in 2019 under the supervision of Eduardo Meneses. During this second iteration, Meneses used the students’ feedback to improve the building instructions, tracking common mistakes that happened due to gaps in the documentation. The official T-Stick Github repository can be accessed at https://github.com/IDMIL/TStick.

Fig. 5.
figure 5

Prynth board (https://prynth.github.io/).

Full size image

6.2 Prynth

Prynth is a framework for the development of self-contained DMIs created by Ivan Franco at IDMIL and released to the public in September 2016. It started as a set of tools and procedures for embedded DMIs but soon evolved to become an open-source DMI development kit.

The framework is composed of hardware and software components that, once assembled, form a compact base system for sensor signal acquisition and audio synthesis. The sensor signal acquisition is made through a series of daughter-boards that sit on top of the Raspberry Pi, allowing for the direct connection of up to 80 analog sensors and 48 digital sensors. The audio engine is based on the programming language SuperCollider, and the user can interactively program new synthesis and interaction algorithms through a browser-based text editor.

Prynth is a tool for intermediate DMI developers. It assumes a considerable familiarity with electronics assembly and computer programming. The user is responsible for tasks such as PCB manufacturing, soldering components, installing custom Linux images, flashing micro-controller firmware, and programming the DMI interaction and audio synthesis in SuperCollider.

The project’s website contains a section titled create, divided into an overview of the framework, downloads, and documentation.

The downloads section contains links to the PCB board designs (distributed both in Eagle and Gerber file formats), the microcontroller software, and a ready-to-use Linux image file for the Raspberry Pi. Those that which to install Prynth on top of a previous Linux system can refer to the raw installation instructions on the project’s Github repository.

The documentation is written in the form of a step-by-step cookbook, divided into chapters that include PC board assembly, flashing software/firmware, and the operation of Prynth’s custom software to control the signal acquisition system and interact with SuperCollider. Finally, a set of “Hello World” examples show the path from connecting sensors to mapping them to SuperCollider audio examples. All of this information is relayed through text, screenshots, and high-resolution macro photography.

Prynth was covered in articles by specialized online publications such as Fact Magazine, Hackaday, and Synthopia, which triggered the quick growth of a user community. Some of those users reported having successfully built their first DMIs, much of it due to the quality of the documentation and support materials found on the project’s website. Other users have also reported successfully modifying parts of Prynth to best suit their particular needs.

The Prynth webpage can be found at https://prynth.github.io/.

7 Open Source DMI

In this section, we discuss the concept of Open Source DMI and present the checklist that we hope can help to achieve a basic sharing structure for DMI projects.

7.1 Definition

An Open Source DMI is a digital musical instrument with a well-documented project, that is easy to reproduce or modify, accessible in terms of cost and components availability, and is obtainable publicly for any use. To this end, we have started developing a checklist that attempts to generate a comprehensive set of elements to support the process of replication.

7.2 Checklist

The checklist consists of a set of questions based on each step of the DMI development process and the previously discussed challenges, especially inspired by the OSHWA certification checklist:

General Aspects

  • Is there a structured and organized website concentrating all the documentation and information about the project?

  • Is there an open communication channel with the developers or makers to ask questions during the making process? (e.g., forum, e-mail)

  • Is there a clear overview architecture diagram with the description of each part’s details?

  • Is there a description of the expected skills and resources required to realize the project?

  • Does the documentation highlight critical, uncertain, difficult, or potentially confusing parts of the process of realization?

Hardware (Mechanical Structure and Electronics)

  • Is the hardware made out of an accessible consumer product? (e.g., Bela, LEGO, Arduino, hardware synthesizers)

  • Are the original design files for the instrument’s structure and electronic circuits publicly available (e.g., CAD files, schematics, technical diagrams)?

  • Is the bill of materials publicly available and accessible? (e.g., links, prices, detailed descriptions)

  • Are there any instructions and explanations about the process of making the instrument’s physical structure and electronic circuits publicly available? (e.g., wiki, Instructables, tutorials, images or videos about the process)

  • Is there a source code version control publicly available for all the hardware and the mechanical structure? (e.g., git or SVN repositories)

  • Are the design files licensed in a way that others may reproduce or build upon them? (e.g., Creative Commons)

  • Is the construction and hardware design under an open-source license (e.g., CERN, TAPR, Solder Pad) so others may reproduce or build upon it?

Software

  • Is the software/firmware publicly available and with an open-source license?

  • Is there a source code version control publicly available for the software? (e.g., git or SVN repositories)

  • Does the instrument rely on an easily accessible proprietary software? (e.g., Max/MSP, Ableton Live, VST Plugins)

  • Are the support and configuration files publicly available? (e.g., mappings, software configuration diagrams, audio samples, presets, DAW project files)

  • Are the presets, files, and projects licensed in a publicly available license (e.g., copyrighted audio samples)?

  • Does the instrument use a standard communication protocol? (e.g., MIDI, OSC or Libmapper)

8 Future Works

By reflecting upon the digital musical instrument design and development process, we expect to improve the proposed checklist to achieve a certification level for DMI projects. Depending on the content and on how well the documentation is presented, the project would receive the title of Open Source DMI. We could also think on a scale that could classify the level of potential replication of the DMI. For example, if there were pieces of evidence that a different group apart from the initial design team replicated the DMI, that project would receive more points. The implementation of this system still an open question. However, we hope to have given an important step towards a more reliable way of sharing DMI projects to strengthen our community.

9 Conclusion

This paper is a step towards an actual standard to share DMI open-source project. The main contribution of this paper is to continue the discussion about sharing DMI designs around the world and how the process can be improved. We focused on discussing different challenges concerning the sharing of physical artifacts online. We presented an emerging checklist in the form of questions that serves as a starting point for designers interested in sharing their projects. Refining this list can lead to a more formal certification process that provides a universal set of standards. At the same time, since the nature of such projects are fundamentally creative in nature, care should be taken to tread the fine line between enforcing standards for the sake of reproducibility and hindering the creative process.