Removing duplicates from Spotify Playlist

[ottic]

Hello,

First of all thank you for the great information on this forum and on the N8N site. It helped me a lot to install N8N on a digital ocean server and creating my first Spotify workflow that I already uploaded.

I have a lot of other ideas for manipulating Spotify Playlists but sadly my Javascript knowledge is very poor and thats why I decide to ask the following maybe stupid question. I tried now for hours but could not find a solution.

I want to remove duplicate entries from a Spotify playlist.

First I read the playlist and extract the Track URIs in a second step with a function, so that I have a result like this:

[
{
“uri”: “spotify:track:3zMu2CI9WkfWPVKJFWlSIf”
},
{
“uri”: “spotify:track:7M6DmpVVG1LnUEZovp7csv”
},
{
“uri”: “spotify:track:3zMu2CI9WkfWPVKJFWlSIf”
},
{
“uri”: “spotify:track:7M6DmpVVG1LnUEZovp7csv”
}
]

Now i tried for hours to remove the duplicate URIs with the Function Code from your Javascript Snippets page but get totally stuck

The result should be like this:

[
{
“uri”: “spotify:track:3zMu2CI9WkfWPVKJFWlSIf”
},
{
“uri”: “spotify:track:7M6DmpVVG1LnUEZovp7csv”
},
]

I hope that one of the experienced users here can help me. Thank you very much in advance.

[ivov]

If you receive an array and would like to return it deduped from a function node, you can filter with a set. You may need to adjust this example based on your workflow.

const tracks = [ 
  { uri: "spotify:track:3zMu2CI9WkfWPVKJFWlSIf" },
  { uri: "spotify:track:7M6DmpVVG1LnUEZovp7csv" },
  { uri: "spotify:track:3zMu2CI9WkfWPVKJFWlSIf" },
  { uri: "spotify:track:7M6DmpVVG1LnUEZovp7csv" }
]

const seen = new Set();

return tracks.filter(t => {
  if (seen.has(t.uri)) return false;
  seen.add(t.uri);
  return true;
}).map(t => ({ json: t }));

This may be useful to read:

Key concepts | n8n Docs

[ottic]

Great! Thanks a lot for your help. This is working now also, when I add the items like this:

const tracks = [ 
  { uri: items[0].json.uri },
  { uri: items[1].json.uri },
  { uri: items[2].json.uri },
  { uri: items[3].json.uri }
]


const seen = new Set();

return tracks.filter(t => {
  if (seen.has(t.uri)) return false;
  seen.add(t.uri);
  return true;
}).map(t => ({ json: t }));

Now I need to get some hint how to get all item objects into the const automatically. I tried already some ideas like using the $items(“Previous Node”) array and also counting the items and do a “for” loop, but no luck until now. Another little help would be so great.

I post the complete workflow here, so that you get an idea of the whole thing:

[ivov]

Try this:

const seen = new Set();

return items.filter(i => {
  if (seen.has(i.json.uri)) return false;
  seen.add(i.json.uri);
  return true;
});

Note that the constant tracks was removed - it was meant as a stand-in.

items is the array of objects passed from node to node. It is available always by default inside the scope of the Function node, and it must be of type Array<{ json: object }> as shown in the link above.

[ottic]

Thanks again for your great help. I think i was a little bit to euphoric when it was working with my little 4 track playlist. After some further testing there are still some issues.

First Issue:

If I have more than 2 duplicates in a playlist like:

Track A
Track A
Track A
Track B

It should remove 2 times Track A but it only removes it once so that the result is:

Track A
Track A
Track B

The result should be:

Track A
Track B

Second Issue:

When i delete tracks from a playlist that is a bit longer in my case about 200 tracks i get an error message:

ERROR: Spotify error response [400]: Could not remove tracks, please check parameters.

I searched the web and found some other people with this issue. I think the reason could be, that there is missing parameter called “positions” that can not be transferred through the Spotify Integration. You can also see it on the Spotify API website here:

Remove Playlist Items | Spotify for Developers

Looking forward to your comments and thanks again for the help so far.

[ivov]

From a quick review, it seems Spotify tracks should be removed not with track.uri but with track.linked_from.uri. In addition, the Spotify node source has 0 hardcoded for positions, which might also be an error. Thanks for reporting this. Added to my TODO list.

As an aside, the snippet in post #4 should remove all duplicates, since a set only allows for unique values. Best to revisit once removal is fixed.

Edit: On second thought, if you are looking to remove duplicates, you should select the duplicates and remove those. Currently you are selecting uniques and removing those, which means that if a track exists three times, you are always removing it only once. Simply invert true and false in the snippet to select duplicates.

const seen = new Set();

return items.filter(i => {
  if (seen.has(i.json.uri)) return true;
  seen.add(i.json.uri);
  return false;
});

The removal bug will still need to be looked at.

[ottic]

Great! Thanks a lot.

[deevee]

@ottic Did you ever get this working? I tried this workflow you published, but it seems to just delete everything in the playlist instead of removing the duplicates. I think it’s because using the delete-track-by-uri call results in the deletion of every track with that uri in the playlist.

One approach to fix that is to completely replace the playlist with the deduplicated tracks using the PUT method (docs). AFAICT n8n doesn’t support this endpoint yet, but you can work around that by calling the Spotify API directly with an HTTP Request node. You’ll need to set up plain Oauth2 Api credentials using the info from your Spotify dev portal app.

Authorization URL: Anmelden - Spotify
Access Token URL: https://accounts.spotify.com/api/token
Scope: playlist-modify-public playlist-modify-private playlist-read-private playlist-read-collaborative
Authentication: Header

It should look something like this:

I’ve got more scopes in my setup, because I use this for other workflows too.

And then for the actual nodes, try this workflow:

{
  "nodes": [
    {
      "parameters": {
        "authentication": "oAuth2",
        "requestMethod": "PUT",
        "url": "https://api.spotify.com/v1/playlists/REPLACEME/tracks",
        "options": {},
        "bodyParametersUi": {
          "parameter": [
            {
              "name": "uris",
              "value": "={{$json.uris}}"
            }
          ]
        }
      },
      "name": "Replace playlist with deduplicated tracks",
      "type": "n8n-nodes-base.httpRequest",
      "typeVersion": 1,
      "position": [
        850,
        300
      ],
      "credentials": {
        "oAuth2Api": "spotify oauth"
      }
    },
    {
      "parameters": {
        "resource": "playlist",
        "operation": "getTracks",
        "id": "spotify:playlist:REPLACEME",
        "returnAll": true
      },
      "name": "Get playlist to deduplicate",
      "type": "n8n-nodes-base.spotify",
      "position": [
        450,
        300
      ],
      "typeVersion": 1,
      "executeOnce": true,
      "credentials": {
        "spotifyOAuth2Api": "Spotify"
      }
    },
    {
      "parameters": {
        "functionCode": "if (items.length == 1 && Object.keys(items[0].json).length == 0) {\n  return [];\n}\n\nconst seen = new Set();\nitems.forEach(i => seen.add(i.json.track.uri));\n\nreturn [{json: {uris: [...seen]}}];\n"
      },
      "name": "Extract unique track URIs",
      "type": "n8n-nodes-base.function",
      "position": [
        650,
        300
      ],
      "typeVersion": 1,
      "alwaysOutputData": true
    }
  ],
  "connections": {
    "Get playlist to deduplicate": {
      "main": [
        [
          {
            "node": "Extract unique track URIs",
            "type": "main",
            "index": 0
          }
        ]
      ]
    },
    "Extract unique track URIs": {
      "main": [
        [
          {
            "node": "Replace playlist with deduplicated tracks",
            "type": "main",
            "index": 0
          }
        ]
      ]
    }
  }
}

Just copy and paste that into a workflow. Replace “REPLACEME” with your playlist ID.

This worked for me. If it works for you, please update your published workflow with this, so others trying it don’t have their playlists wiped by accident.

[ottic]

Thanks a lot for your message. You’re totally right, the remove duplicates workflow was my first input here and it was not working accurately. I think because of the bug mentioned by “ivov” above. I removed the whole workflow for now.

Your solution is working, but only for playlists with up to 100 duplicates. If you use it on a playlist with more than 100 duplicates you get the following error message:

HTTP Code: rejected
“status”:400,“message”:“You can add a maximum of 100 tracks per request.”

It should be possible to fix this with splitting the input.