Every neural audio codec in use today encodes music at a constant rate: the time--frequency plane is cut into units whose shape is fixed by the architecture before any audio is heard, and none of these systems can change the shape of the units it encodes. This thesis asks whether a representation of music can be built the other way round, by letting the signal itself decide how its own plane is divided. The substrate that makes the question expressible is a graph. Contracting neighbouring regions into a single unit --- coarsening --- is a structural operation that a graph supports natively and a grid cannot express, and it is the reason graph neural networks are used here. Until now they had reached audio only for discriminative tasks, where nothing has to be reconstructed; whether a graph representation of music can be inverted back into sound was a question that, to our knowledge, had not been posed. The thesis answers it in the affirmative and measures what the answer costs. A spectrogram read as a graph and coarsened by its own signal reconstructs better, at a matched number of tokens, than a content-blind grid sharing every other component, and reaches a quality the grid does not match with seven times as many tokens --- with a model of \num{654850} parameters, a factor of twenty-seven below the closest published system. Fifty-two listeners in a blind test prefer it to the rate-matched control, and their ratings validate the perceptual metric the comparisons rest on. What decides the outcome is not adaptivity as such but the merge criterion: coarsening paid when the cost assigned to a merge approximated the error that merge causes, and lost when it did not. Inside the trained model, the tokens that cover silence collapse to a single connected region per clip, with two and a half times less variance than those of the note: the model arrives at a codeword for silence without being asked for one, and that is where the rate is saved. The efficiency claimed is a property of the representation and is counted in tokens, not in transmitted bits; the cost of the partition is measured and reported beside it. What this work places is a first stone: the first reconstruction model built on the coarsening of a graph.
An efficient latent representation of music based on graph neural networks
CHESSA, GIOVANNI
2025/2026
Abstract
Every neural audio codec in use today encodes music at a constant rate: the time--frequency plane is cut into units whose shape is fixed by the architecture before any audio is heard, and none of these systems can change the shape of the units it encodes. This thesis asks whether a representation of music can be built the other way round, by letting the signal itself decide how its own plane is divided. The substrate that makes the question expressible is a graph. Contracting neighbouring regions into a single unit --- coarsening --- is a structural operation that a graph supports natively and a grid cannot express, and it is the reason graph neural networks are used here. Until now they had reached audio only for discriminative tasks, where nothing has to be reconstructed; whether a graph representation of music can be inverted back into sound was a question that, to our knowledge, had not been posed. The thesis answers it in the affirmative and measures what the answer costs. A spectrogram read as a graph and coarsened by its own signal reconstructs better, at a matched number of tokens, than a content-blind grid sharing every other component, and reaches a quality the grid does not match with seven times as many tokens --- with a model of \num{654850} parameters, a factor of twenty-seven below the closest published system. Fifty-two listeners in a blind test prefer it to the rate-matched control, and their ratings validate the perceptual metric the comparisons rest on. What decides the outcome is not adaptivity as such but the merge criterion: coarsening paid when the cost assigned to a merge approximated the error that merge causes, and lost when it did not. Inside the trained model, the tokens that cover silence collapse to a single connected region per clip, with two and a half times less variance than those of the note: the model arrives at a codeword for silence without being asked for one, and that is where the rate is saved. The efficiency claimed is a property of the representation and is counted in tokens, not in transmitted bits; the cost of the partition is measured and reported beside it. What this work places is a first stone: the first reconstruction model built on the coarsening of a graph.| File | Dimensione | Formato | |
|---|---|---|---|
|
tesi_chessa.pdf
accesso aperto
Dimensione
3.61 MB
Formato
Adobe PDF
|
3.61 MB | Adobe PDF | Visualizza/Apri |
The text of this website © Università degli studi di Padova. Full Text are published under a non-exclusive license. Metadata are under a CC0 License
https://hdl.handle.net/20.500.12608/115828