I'm excited to finally share something I have been working on for the last year🪄 Holly+
I am releasing Holly+ in collaboration with Never Before Heard Sounds, the first tool of many to allow for others to make artwork with my voice, and will distribute ownership of my digital likeness through the creation of the Holly+ DAO 👭
My voice is precious to me! It is 1 of 1 🥰
Voice Models, in combination with machine learning technology, already allow for anyone to clone a voice to generate music and media, and the opportunities and complications inherent to these techniques will only intensify!
This development raises novel questions about voice ownership that I think can be addressed by DAO governance 🤝
I plan to decentralize access, decision making and profits made from my digital twin, Holly+ 👭
Profits made from the commercial usage of my digital likeness will be ingested by the DAO, who are able to vote on new tool creation, and approve official usage of my twin, Holly+ 🧞♀️
I will soon be launching a Zora auction house to sell approved works made with my likeness 👩⚖️
Who am I?
I'm an artist and composer 👩💻 who has been working with machine learning for many years. My last album PROTO(4AD,2019) was the first to utilize singing neural networks, and I completed my Doctorate at Stanford's Center for Computer Research in Music and Acoustics, where my research focus was on the interplay between machine learning and the voice, and the implications of this technology for IP and vocal sovereignty👩🔬
Some AI models already know who I am! Here are some images spawned from my likeness using OpenAI's CLIP model 🤯
Here are some links to find me online:
Wikipedia / Foundation / Twitter / Instagram
Holly+ Voice Tool
The first tool, a custom voice instrument and website by Never Before Heard Sounds, allows for anyone to upload polyphonic audio and receive a download of that music sung back in my distinctive processed voice.
Further tools are in development to allow for spoken and sung phrases, and image generation, to be released later this year.
There is an upload limit of 5 minutes for audio files ⏲️
We do not retain a copy of any original audio uploaded to the tool, but we will archive copies of the output audio so you can share them!
You are free to use the material generated as you see fit, and accreditation is always appreciated! 😎
What is a Voice Model?
A Voice Model is a deep neural network that can generate raw audio of an individual voice. The network is trained on recorded speech and singing from the target voice, and can be interacted with in various ways, from text-to-speech applications to more complex interactions such as audio style transfer, where audio from one voice can be converted to resemble the target voice, a kind of vocal puppetry 🤖
The recent introduction of projects like DeepMind’s Wavenet, Google’s Tacotron and others have advanced the field of voice generation sufficient to make me confident that generating convincing spoken and sung voices will soon become standard practice for artists and other creatives, as presaged by the popularity of celebrity vocal deep fakes already found all over YouTube.
Not all vocal models are equal. While it may be possible to generate an approximation of a voice by training a neural network from freely available audio found online, the deeper and more detailed the model is, the wider the range of expression available to create with 🖌️
While for the time being the perceived difference between a bootleg or official voice model may be quite small, as our ability to produce more detailed and convincing voice generation evolves, so too will the need for comprehensive, high fidelity vocal training data, as well as the urgency of discerning provenance.
For this reason I believe that there will be demand for official, high fidelity, vocal models of public figures, so why not experiment with my own? 🗣️
These models will guarantee the most realistic possible generated speech and song, and also provide a means for certifying officially approved usage of the voice in question.
Fidelity and range will be achieved by providing training input of the celebrity voice across different epochs of their lives, as a voice changes with the body, and also incorporate detailed aspects such as signature vocal habits and artificial effects 🥸
Voice Model Rights
Existing legal precedent protects public figures against artists or brands commercially appropriating their vocal likeness. There are two regularly cited cases arguing this point.
First, in Midler v. Ford Motor Co (1988), Bette Midler sued and won her case against Ford Motor Company, who had hired one of her backing singers to impersonate her delivery on her hit “Do You Want to Dance” after Midler herself had refused the offer. You can hear the original song here, and the convincing re-enactment here 🤔
Second, in Waits v. Frito-Lay (1992) , the artist Tom Waits sued and won a case against Frito-Lay for their vocal impersonation of him in a Doritos radio commercial. Significantly, the impersonation was not referencing a song he previously had any association with. This suggests that beyond practices of digital sampling of existing recordings or vocal reenactment of existing repertoire, “some stylistic aspects of the voice are definable, extractible, and defensible against unauthorized use under a right of publicity theory.”
While we are yet to see legal cases in relation to the commercial exploitation of a vocal likeness utilizing deep learning, these precedents suggest that public figures will retain exclusive rights to the exploitation of their vocal likeness for commercial purposes.
While a public figure is protected from unlicensed commercial exploitation of their vocal likeness, there are also great benefits to offering the public open source access to a voice model, ensuring that all derivative works created through experimentation in the voice would also be open source and not commercially exploitable 👩🏫
Through providing artists and researchers access to the voice under these terms, it increases the likelihood of the voice’s prominence online, which in turn only contributes to the value of the original voice model itself.
This approach supports the “My Collectible Ass” principle, advocated for by ZORA and originally proposed by theorist McKenzie Wark. This principle states that the more prominent and visible/audible a work of art is, the more valuable the certified original becomes.
Open sourcing the voice model leaves open the possibility for its non commercial use, however only the owners and stewards of the voice model would be able to certify and profit from derivative works created from it, which feels fair 🤝
Holly+ DAO
DAO (Decentralised Autonomous Organization) stewardship of my voice model rights makes sense for many reasons.
It is an idea I have been interested in for a long time! I dedicated a song to the idea on my album Platform(4AD,2015) 🧮
A DAO of stewards of the Holly+ voice model can vote on minting and certifying new works created from the voice model. As token holders, they are incentivized to only certify or license new works that contribute to the value of the voice, and do not dilute that value through the production of bad art or negative associations. Only a scarce number of official artworks will be minted using the voice model, and DAO token holders are entitled to a cut of profits from resales of those works in perpetuity 🤝
The Holly+ model creates a virtuous cycle. I release tools to allow for the creative usage of my likeness, the best artworks and license opportunities are approved by DAO members, profit from those works will be shared amongst artists using the tools, DAO members, and a treasury to fund further development of the tools.
In the event of my death, I feel more comfortable with distributed ownership of the rights to my voice model among a DAO of stewards who are invested in maintaining the value and reputation of my voice than the rights being passed down to someone less familiar with the values and standards associated with my work. There is precedent for music catalog rights being passed down to family estates whose best interest is in making as many short term gains from those rights as possible. My hope is that decentralizing decision making across a DAO of stewards will encourage a more delicate stewardship over those rights through debate and shared interest!
OpenLaw Tribute DAO Framework
The Holly+ DAO will make use of the OpenLaw Tribute DAO framework. You can learn more about OpenLaw, and their instrumental role in legally recognizing DAO's in Wyoming, on our Interdependence podcast here:
Holly+ DAO tokens
I will create ERC-20 VOICE tokens, airdropped to collectors of my art, friends and family of the project, and other artists selected to participate in using the Holly+ voice to create new works.
The ERC-20 VOICE tokens will come to represent voting shares in Holly+ DAO.
DAO token holders will vote to mint new NFTs of derivative works generated by Holly+ voice, and the DAO will be entitled to a % cut of profits made from those sales in perpetuity.
The value of the token will be derived from access to governance decisions in stewarding the voice model and minting new works, royalties from any new works minted, secondary sales of all works minted, and eventually any commercial licensing of the Holly+ voice on or off chain, once we are able to reach our goal of recognizing the DAO as a legal entity.
NFT Minting
Soon, artists will be able to submit media created with the voice for DAO approval through a ZORA auction model.
Should the DAO approve the work, an NFT will be minted, with 50% of initial and secondary sales shared with the creator of the new work. 40% of profits will be distributed amongst DAO members, and 10% will go to me directly, for rent!
NFT Minting Scenario
Artist X produces a song using the Holly+ voice model.
Artist X uploads the song online, and submits the song as a proposal to the DAO through a public interface.
VOICE token holders vote to mint the song as an appropriate or inspiring usage of the Holly+ voice.
An NFT of the song is minted by the DAO, with 50% of sales generated going to the artist, and 40% of sales going to DAO members, and 10% reserved for me.
Roadmap
2021
Launching of the interface to easily be able to make new works with the Holly+ voice model ✅
Open sourcing of the vocal model to encourage experimentation and development ⏳
The first minting of NFTs using the Holly+ voice model from select collaborators, with 40% profits from sales being absorbed back into the DAO ⏳
Further tools for creation using the Holly+ voice ⏳
2022
The establishment of Holly+ VOICE DAO LLC, offering profit splits with members for commercial usage of the voice model in on-chain and off-chain scenarios ⏳
Further development of the Holly+ voice model, and new tool development funded by DAO proceeds and voted upon by DAO members ⏳
Partners & Advisors
Never Before Heard Sounds https://heardsounds.com
Zora https://zora.co/
OpenLaw https://www.openlaw.io/
RVNG Intl. https://igetrvng.com/
Eternal thank yous Mat Dryhurst, Reuben Bramanathan, Matt Liston, Trent McConaghy, Cooper Turley, Ashleigh Schap, Trevor McFedries and Simon de la Rouviere for your patient advice! Thank you Ezra Miller, Mikey Joyce and Andrés Mañón for the beautiful artwork and photography! 💓
Never Before Heard Sounds
Never Before Heard Sounds, founded by Chris Deaner & Yotam Mann, makes machine learning instruments and expressive audio tools.
For Holly+ we trained a custom model on multiple hours of Holly Herndon’s isolated vocal stems to create a generative instrument that retains the pitches and rhythms of a user-uploaded audio file, but adds textures and timbres learned from the training set. The Holly+ model runs on our dedicated GPU backend that we use for all our cloud transformations. In addition to the cloud processor, all our models (including Holly+) can be performed live on a real-time hardware instrument we are developing.
Listen to our discussion with Never Before Heard Sounds on the Interdependence podcast
Zora
Zora is a universal media registry protocol. It’s a way for creators to publish creative media, earn money on their work, and have others build and share what they create. Because this media is based on a protocol, all this happens without the lock-in of social media platforms.
Read what Zora does: https://zora.co/manifesto
Learn how Zora does it: https://zora.engineering/whitepaper
Listen to our discussion with Zora on the Interdependence Podcast:
Legal disclaimer
During my lifetime, I will exclusively retain the right to do whatever I want with my physical voice! This project exclusively concerns Holly+, my digital vocal twin 😘