[GH-ISSUE #648] Integrate with Dropbox and Google Drive #464

Closed
opened 2026-02-26 02:33:17 +03:00 by kerem · 7 comments
Owner

Originally created by @phanan on GitHub (Sep 15, 2017).
Original GitHub issue: https://github.com/koel/koel/issues/648

Just leaving this here as a reminder and to welcome ideas/contributions.

Originally created by @phanan on GitHub (Sep 15, 2017). Original GitHub issue: https://github.com/koel/koel/issues/648 Just leaving this here as a reminder and to welcome ideas/contributions.
Author
Owner

@natrod commented on GitHub (Nov 9, 2017):

@phanan Great Project! I would love to contribute to the effort during my free time

<!-- gh-comment-id:343170760 --> @natrod commented on GitHub (Nov 9, 2017): @phanan Great Project! I would love to contribute to the effort during my free time
Author
Owner

@DevinNorgarb commented on GitHub (Jan 30, 2018):

@phanan old thread, I know, but I am successfully using Google Drive as a CDN for Koel.

Simply put, you can mount Google Drive on an Ubuntu VPS which is what I have done, then just reference the mountpoint as the root media path and voila, all my music is streamed directly from Google Drive.

If there is any interest I 'd be happy to explain in depth.

<!-- gh-comment-id:361700322 --> @DevinNorgarb commented on GitHub (Jan 30, 2018): @phanan old thread, I know, but I am successfully using Google Drive as a CDN for Koel. Simply put, you can mount Google Drive on an Ubuntu VPS which is what I have done, then just reference the mountpoint as the root media path and voila, all my music is streamed directly from Google Drive. If there is any interest I 'd be happy to explain in depth.
Author
Owner

@phanan commented on GitHub (Jan 30, 2018):

@DevinNorgarb Not an old thread, so no worries 😄
As I'm not familiar with GDriver as well as the whole mounting idea, can you share a bit more about e.g. the performance? Does scanning work just seamlessly?

<!-- gh-comment-id:361711586 --> @phanan commented on GitHub (Jan 30, 2018): @DevinNorgarb Not an old thread, so no worries 😄 As I'm not familiar with GDriver as well as the whole mounting idea, can you share a bit more about e.g. the performance? Does scanning work just seamlessly?
Author
Owner

@DevinNorgarb commented on GitHub (Feb 1, 2018):

Yes absolutely, I use google-drive-ocamlfuse to mount it.

Before you install the package just head over to your
Google Developer API's account, select a new project if you haven't, then find the Google Drive API under their API Libary and enable the API.

After following the instructions to mount the drive, if all is successful, you should have google drive mounted as an external harddrive, you can browse through it like you would with any other directory in Linux as well as rename, delete, and copy files to and from it.

They make use of Readahead which streams and caches media instead of downloading each song every time.

I have found using Koel to scan and stream the media directory works perfectly and imports everything without an issue.

When I have time I will run some bench marks for you.

<!-- gh-comment-id:362183442 --> @DevinNorgarb commented on GitHub (Feb 1, 2018): Yes absolutely, I use [google-drive-ocamlfuse](https://github.com/astrada/google-drive-ocamlfuse) to mount it. Before you install the package just head over to your [Google Developer API's ](https://console.developers.google.com/apis/library) account, select a new project if you haven't, then find the [Google Drive API](https://console.developers.google.com/apis/library/drive.googleapis.com) under their API Libary and enable the API. After following the instructions to mount the drive, if all is successful, you should have google drive mounted as an external harddrive, you can browse through it like you would with any other directory in Linux as well as rename, delete, and copy files to and from it. They make use of Readahead which streams and caches media instead of downloading each song every time. I have found using Koel to scan and stream the media directory works perfectly and imports everything without an issue. When I have time I will run some bench marks for you.
Author
Owner

@DevinNorgarb commented on GitHub (Dec 28, 2018):

@phanan

Hi guys
Here is an update on the Google Drive integration.

How I went about doing the integration:

Importing

Created an Import sidebar title with Google Drive as an option

On the Import page we can use Googles file picker to authenticate ourself and select individual or multiple tracks/directories.
screenshot from 2018-12-28 21-06-34
screenshot from 2018-12-28 21-01-21

Once we have selected our files we to import the picker fires an On Select event which is we catch an array of objects, each object containing detailed info on the track.

I pass this to a controller where I loop though each record, grab what I need then pass this on to a queue job.

This is where it got a bit tricky. I needed all the metadata which Koel requires to build the Track, Album, Artist, Cover Art and other info, but this cannot be done using http, so passing the URL Google provides
is with to access the track won't work which is explained here by its creator, James Heinrich.
getID3 using remote urls.

An option is to download the first 32kb of the track, which might or might not be enough to get all the info out so just doing a full download is what I choose.

I did this by opening a stream with the url google provides, the placing this in the /tmp dir scanning it with getID3, which gives us everything we need to pass to \App\Models\File to create new instance.

Similar to way we check a property named $song->s3_params I check for $song->cdn == 'google_drive' then save the ID google provides for the track since the file is not going to remain in the /tmp much longer.
This ID (saved in the db as the songs path)will be refrenced later to Identify and stream the track.
I then call `$file->sync() which saves the song and all it's metadata.

We can just unlink the temp file since it's served it purpose.

Streaming

In the SongControllers play() method I check for $song->cdn == 'google_drive' again

` if ($song->s3_params) {
return (new S3Streamer($song))->stream();
}

    if ($song->cdn == 'google_drive') {
        return (new GoogleDriveStreamer($song))->stream();
    }`

This is Similar to the S3 check.

If true a new GoogleDriveStreamer is instantiated.

For convenience here I add Google Drive as a Flysystem adapter

After creating a stream I pass this to laravels response->stream()

   public function stream()
    {
        $fs = \Storage::disk('google')->getDriver();
        $metaData = $fs->getMetadata($this->song->path);
        $stream = $fs->readStream($this->song->path);

        if (ob_get_level()) {
            ob_end_clean();
        }

        return response()->stream(
                    function () use ($stream) {
                        fpassthru($stream);
                    },
                    200,
                    [
                        'Content-Type' => $metaData['type'],
                        'Content-disposition' => 'attachment; filename="' . $metaData['path'] . '"',
                        "Content-Length" => $metaData['size'],
                        "Last-Modified" =>  "Sat, 26 Jul 1997 00:00:00 GMT",  // leverage browser caching
                        "Accept-Ranges" => 'bytes'
                    ]
                );
    }

This streams the track, exactly as expected with cover art, LastFM info and is cachied, the full monty.
The streaming is milissconds slower than using a local file.

Feedback would be great on how I went about this.

Imported Tracks!
screenshot from 2018-12-28 22-36-14

notes:

  • I have accomplished the same thing for dropbox.
  • I only pay $9.99 per month for 2 TB of storage which is brilliant.
  • I run Koel on a cloud VPS so datatransfer is essentially unlimited.
  • This works without having to download the track just to get the getID3 tags, but with this you lose out on a ton on song info. Just not cool. (Google does provide metadata but this did not satify me).
  • Importing big libaries on limited data / internet speed would not be viable so a solution/optimisation is needed there

`

<!-- gh-comment-id:450427231 --> @DevinNorgarb commented on GitHub (Dec 28, 2018): @phanan Hi guys Here is an update on the Google Drive integration. How I went about doing the integration: ## Importing Created an `Import` sidebar title with `Google Drive` as an option On the Import page we can use Googles [file picker](https://developers.google.com/picker/) to authenticate ourself and select individual or multiple tracks/directories. ![screenshot from 2018-12-28 21-06-34](https://user-images.githubusercontent.com/12491966/50525797-6ed94780-0ae6-11e9-982d-2a757fc3fa45.png) ![screenshot from 2018-12-28 21-01-21](https://user-images.githubusercontent.com/12491966/50525798-6ed94780-0ae6-11e9-84ad-a906ce22cf20.png) Once we have selected our files we to import the picker fires an `On Select` event which is we catch an array of objects, each object containing detailed info on the track. I pass this to a controller where I loop though each record, grab what I need then pass this on to a queue job. This is where it got a bit tricky. I needed all the metadata which Koel requires to build the Track, Album, Artist, Cover Art and other info, but this cannot be done using http, so passing the URL Google provides is with to access the track won't work which is explained here by its creator, James Heinrich. [getID3 using remote urls](https://www.getid3.org/phpBB3/viewtopic.php?t=1605). An option is to download the first 32kb of the track, which might or might not be enough to get all the info out so just doing a full download is what I choose. I did this by opening a stream with the url google provides, the placing this in the `/tmp` dir scanning it with getID3, which gives us everything we need to pass to \App\Models\File to create new instance. Similar to way we check a property named $song->s3_params I check for $song->cdn == 'google_drive' then save the ID google provides for the track since the file is not going to remain in the `/tmp` much longer. This ID (saved in the db as the songs path)will be refrenced later to Identify and stream the track. I then call `$file->sync() which saves the song and all it's metadata. We can just unlink the temp file since it's served it purpose. ## Streaming In the `SongControllers` play() method I check for $song->cdn == 'google_drive' again ` if ($song->s3_params) { return (new S3Streamer($song))->stream(); } if ($song->cdn == 'google_drive') { return (new GoogleDriveStreamer($song))->stream(); }` This is Similar to the S3 check. If true a new GoogleDriveStreamer is instantiated. For convenience here I add [Google Drive as a Flysystem adapter](https://github.com/nao-pon/flysystem-google-drive) After creating a stream I pass this to laravels response->stream() ``` public function stream() { $fs = \Storage::disk('google')->getDriver(); $metaData = $fs->getMetadata($this->song->path); $stream = $fs->readStream($this->song->path); if (ob_get_level()) { ob_end_clean(); } return response()->stream( function () use ($stream) { fpassthru($stream); }, 200, [ 'Content-Type' => $metaData['type'], 'Content-disposition' => 'attachment; filename="' . $metaData['path'] . '"', "Content-Length" => $metaData['size'], "Last-Modified" => "Sat, 26 Jul 1997 00:00:00 GMT", // leverage browser caching "Accept-Ranges" => 'bytes' ] ); } ``` This streams the track, exactly as expected with cover art, LastFM info and is cachied, the full monty. The streaming is milissconds slower than using a local file. Feedback would be great on how I went about this. Imported Tracks! ![screenshot from 2018-12-28 22-36-14](https://user-images.githubusercontent.com/12491966/50528201-8bc94700-0af5-11e9-9e15-894669c5f21d.png) ## notes: - I have accomplished the same thing for dropbox. - I only pay $9.99 per month for 2 TB of storage which is brilliant. - I run Koel on a cloud VPS so datatransfer is essentially unlimited. - This works without having to download the track just to get the getID3 tags, but with this you lose out on a ton on song info. Just not cool. (Google does provide metadata but this did not satify me). - Importing big libaries on limited data / internet speed would not be viable so a solution/optimisation is needed there `
Author
Owner

@qwerzl commented on GitHub (Dec 11, 2022):

Is this still a work in progress? Would love to see the feature!

<!-- gh-comment-id:1345474792 --> @qwerzl commented on GitHub (Dec 11, 2022): Is this still a work in progress? Would love to see the feature!
Author
Owner

@phanan commented on GitHub (Jul 26, 2024):

Dropbox integration has been implemented in v7. Google Drive will be added if/when I have the time.

<!-- gh-comment-id:2252903290 --> @phanan commented on GitHub (Jul 26, 2024): Dropbox integration has been implemented in v7. Google Drive will be added if/when I have the time.
Sign in to join this conversation.
No milestone
No project
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference
starred/koel-koel#464
No description provided.