Analysis of Cyborg Mixing and Mastering in Musical Culture:

How Could We Use AI Technology Properly?

 Currently, human resources and technology harmoniously co-exist in society. Although technology has been artificially developed, it has already surpassed us in many ways and will likely continue to progress. A society developed with technological prowess resulted in numerous benefits, but it is also factual that technology has adversely impacted society. 

In particular, the music society has benefited significantly from technology but also suffered setbacks. The current technology utilized by the music industry is mainly artificial intelligence. Concerning AI technology, we should pay attention to AI mixing and mastering human skills, which are essential during the production process Mixing and mastering require professional skills. Still, humanistic nuances and human emotions are included, which is why it takes time, experience, and a process to combine intuition and technology. 

However, artificial intelligence mixing goes through a completely different process than human mixing. Artificial intelligence mixing applies to mixing and mastering music using algorithms that are input to the data. Furthermore, to eliminate the unnatural and mechanical feeling, information on numerous style’s mixing and mastering is collected using big data. Through the collected information, it is once again substituted and applied to music. Then, the mixing and mastering of two different layers are mixed to complete the process. Nevertheless, these processes only take a few seconds to minutes.

However, is the technology leading to artificial intelligence perfect? Artificial intelligence mixing incurs several errors during the mastering process. These days, CDs and vinyl's usability places more significance on the material possession value. However, CDs and vinyl in the music industry were objects to inform and share music in the past. So, the mastering process was based on the overall albums that could be put on CDs and vinyl rather than on individual tracks. This mastering purpose has not changed, and although it means mastering each track, it is more meaningful to connect each track going into the album. In “Machine Learning in Context, or Learning from LANDR,” Sterne and Razlogova inform that “LANDR is designed to work primarily at the track level. For most of our study, it was only possible to master individual tracks on LANDR; there was no album option… LANDR does not frame mastering in terms of albums.”[1] When working on an album’s specific stage, the mastering technique creates a natural flow considering the tone, amplitude, and dynamics between tracks. However, the process of grouping multiple tracks into one album is required to have human nuance, meaning, purpose and instinct. In the end, artificial intelligence mixing and mastering technology deliver results created through a fast process and big data. Still, it does not meet expectations in the album mastering process that requires humanity. 

Another problem comes from the degree of understanding the frequency. From the same journal by Sterne and Razlogova, they mention the problem with the bass frequency: 

One of the issues for people who work in this kind of space is that they cannot hear or properly manage low frequencies… This means that LANDR gets many recordings with bass problems. If there is too much bass, LANDR clamps it down, disciplines it, and makes sure it does not overwhelm the track or blow up the speakers of anticipated future listeners. LANDR cannot tell the difference between a bass drum, a synth bass, a bass drop, a bass voice, a bass clarinet, a bass guitar, or simply "unruly" bass frequencies.[2] 

Like Sterne's opinion, when the low frequency is not in harmony with the music depending on the space, LANDR does not recognize it and relies only on the data entered into the algorithm to produce results. It means that, in the end, the dynamics of the entire track cannot be read, and each instrument and frequency area present in the track is analyzed and processed only with the set values ​​entered in the data. The problem with frequency shows us how AI mixing technology is not perfect for rough tracks, except for well-balanced tracks that have gone through the mixing process. Artificial intelligence mixing and mastering certainly have some benefits. However, due to limitations in mastering albums using artificial intelligence technology and limitations in analyzing tracks, it seems that it is still not enough to fully reproduce human mixing and mastering human nuance, analysis, and originality.

The question that arises here is, how will AI mixing and mastering technology armed with algorithms impact our culture and society? The artificial intelligence mixing and mastering process described above can produce the sound quality and tone that musicians want quicky through numerous data and complex algorithms. However, convenient and straightforward artificial intelligence technology can shift our overall culture. Furthermore, society may be immersed in a culture based on artificial intelligence rather than one created by humans. Cultural change by artificial intelligence technology is currently taking place in the music industry. 

Mixing and mastering using artificial intelligence have already been commercialized and used by numerous musicians and artists, and the results demonstrate high quality. The most representative artificial intelligence mixing and mastering platform is LANDR. An advantage of using LANDR is cost performance. Often, when a musician completes a song using a mastering engineer, it takes time and can be costly. However, LANDR is divided into monthly and annual services using the subscription system. It can be much more economical than requesting a song from an engineer, considering the results’ money and quality. It also requires significant time that musicians who are not economically uncomfortable will pay for an annual service. Sterne and Razlogova also argue that “Musicians may use LANDR for cost savings as a substitute for commercial mastering, as part of a DIY ethos, or as an alternative to spending nothing at all on mastering.”[3] Artificial intelligence mixing and mastering, which provide services based on online platforms such as LANDR, provide many advantages and opportunities for those who make a living through music. Still, the music industry's culture is changing accordingly. 

In “Music Platforms and the Optimization of Culture,” Morris says that “Platforms are a digital evolution of these affordances; one where the display, discovery, search, and consumption of a cultural good all occur through a software search bar and digital database. As a result, new tactics for standing out and new effects on musical cultures are emerging.”[4] Services such as LANDR, based on online platforms, digitize all post-production processes. The mixing and mastering process is done non-face-to-face, but the music is completed through direct consultation and research with musicians and sufficient dialogue. Mixing and mastering is the last part of the production process, and most mixing and mastering engineers decide music’s direction through dialogue with musicians as part of the music business. Moreover, the music's identity and culture are formed through communication. However, the platform’s ‘onlineization’ and the digitization of consumerization can eliminate the culture of interaction and exchange between musicians and engineers in the music community.

The limitations of artificial intelligence mixing and mastering technology and AI blocking the musical culture between musicians and engineers allow us to present an ideal solution, although not a perfect solution. The solution we will propose is the hybrid mixing and mastering method. Hybrid mixing and mastering models are already widely used in the music industry. In addition to platforms such as LANDR, there are also artificial intelligence plug-ins such as Ozone 9 and Neutron 3 made by Izotope. However, unlike LANDR, these are plug-in software and do not mix or master the entire track, but rather the engineer directly grafts and controls the track. In the reading, “Listening without Ears: Artificial Intelligence in Audio Mastering,” by Thomas Birtchnell, he suggests that “One conclusion to draw is that AI in the cultural industry of audio mastering will need to strive toward human-centered algorithm design, encompassing both critical listening and creativity, in collaboration with humans rather than through attempts to replace them.”[5] Birtchnell’s suggestion has a crucial point about human-centered and collaboration between AI and human.[6] Plug-ins such as Ozone 9 and Neutron 3 serve as a framework for the mixing and mastering method that engineers pursue. They present new directions and perspectives on mixing and mastering. It does not infringe on the exchange culture in the music industry; therefore, artificial intelligence technology can preserve culture based on plug-ins with strong hybrid characteristics while expanding mixing and mastering’s direction and diversity.

In conclusion, we can see that artificial intelligence’s technological prowess continues to have limitations. Besides, LANDR’s online platform can shift the existing music industry’s culture. However, if we are to develop the music industry, we need to think in terms of AI and not disregard artificial intelligence technology. By utilizing a hybrid method, artificial intelligence should not gain an advantage over humans but contribute to musical culture and overall human society’s development.

Bibliography

Birtchnell, Thomas. “Listening without Ears: Artificial Intelligence in Audio Mastering.” Big Data & Society, July 2018, doi:10.1177/2053951718808553.

Morris, Jeremy Wade. “Music Platforms and the Optimization of Culture.” Social Media + Society, July 2020, doi:10.1177/2056305120940690.

Sterne, Jonathan, and Elena Razlogova. “Machine Learning in Context, or Learning from LANDR: Artificial Intelligence and the Platformization of Music Mastering.” Social Media + Society, Apr. 2019, doi:10.1177/2056305119847525.

[1]. Sterne, Jonathan, and Elena Razlogova. "Machine Learning in Context, or Learning from LANDR: Artificial Intelligence and the Platformization of Music Mastering." (Social Media + Society, Apr. 2019), doi:10.1177/2056305119847525. 10. 

[2]. Sterne and Razlogova. "Machine Learning in Context, or Learning from LANDR." 14.

[3]. Sterne and Razlogova. "Machine Learning in Context, or Learning from LANDR." 8.

[4]. Morris, Jeremy Wade. "Music Platforms and the Optimization of Culture." (Social Media + Society, July 2020). 8.  

[5]. Birtchnell, Thomas. "Listening without Ears: Artificial Intelligence in Audio Mastering." Big Data & Society, July 2018, doi:10.1177/2053951718808553. 13. 

[6]. Birtchnell. "Listening without Ears." 13.