mirror of
https://github.com/hoppscotch/hoppscotch.git
synced 2026-04-25 16:55:59 +03:00
[GH-ISSUE #870] [feature]: Sync Collections with Git repo #302
Labels
No labels
CodeDay
a11y
browser limited
bug
bug fix
cli
core
critical
design
desktop
discussion
docker
documentation
duplicate
enterprise
feature
feature
fosshack
future
good first issue
hacktoberfest
help wanted
i18n
invalid
major
minor
need information
need testing
not applicable to hoppscotch
not reproducible
pull-request
question
refactor
resolved
sandbox
self-host
spam
stale
testmu
wip
wont fix
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
starred/hoppscotch#302
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "%!s()"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Originally created by @sa- on GitHub (May 13, 2020).
Original GitHub issue: https://github.com/hoppscotch/hoppscotch/issues/870
Is your feature request related to a problem? Please describe.
Would be nice to sync storage over git. My team runs into this problem with Postman and any other web request UI out there - that the web request collections are synced differently than our git repos. Why not just make storage git-friendly
Describe the solution you'd like
Jupyter notebooks do this well. Their UI is populated by a json file, and any interactions or change in state are stored back into that json format, making it play well with git. We should be able to JSON.stringify whatever needs to be stored and have this work like a web request "IDE".
EDIT: In order to flesh out a solution a little more, requests could be stored as
.httpfiles and follow https://www.w3.org/Protocols/rfc2616/rfc2616-sec5.html@liyasthomas commented on GitHub (May 13, 2020):
Syncing with GitHub repo feature is being requested a lot. Pinning this to roadmap 💯
@ukslim commented on GitHub (Jun 29, 2020):
Please don't tie this feature to GitHub specifically. Allow it to use arbitrary git repositories. Some of us use Bitbucket. Some of us run self-hosted git servers.
@zia-devsinc commented on GitHub (Oct 7, 2020):
Waiting eagerly for this feature
@mittalyashu commented on GitHub (Oct 8, 2020):
Maybe we can commit the connections inside the git repository with a dot folder like
.hoppscotchand store collections inside it.(👆 idea can fulfil this too https://github.com/hoppscotch/hoppscotch/issues/870#issuecomment-651063460)
@al3rez commented on GitHub (Oct 30, 2020):
I'm wiling to start this feature, it'd be nice if anyone can give more context. On how we're going to track requests in Git repositories. For example; through Oauth2 and third-parties? I quite new to this service so please enlighten me.
Thanks 🥋
@liyasthomas commented on GitHub (Oct 30, 2020):
As of now, we've a GitHub Oauth app used for signing in to Hoppscotch using GitHub. This could be extended to implement Collections sync. But I too haven't done this before. If you could be of help, I'm all in to implement this.
Currently Collections (and History and Environments) are synced to Firebase as JSON arrays. Basic MVP should allow users to:
@gempir commented on GitHub (Oct 30, 2020):
That's not the point of a git integration. There should be zero oauth or authentication in general. The point is storing it in plaintext files that can be added to a git repository.
So ideally not binary blobs but nicely reviewable files something similar to
.restor.httpfiles maybe https://www.jetbrains.com/help/idea/http-client-in-product-code-editor.html#creating-http-request-filesAlthough JSON Arrays as described above sound fine aswell. Decently human readable in codereview.
The UI could offer a basic commit and push functionality but I think the first goal should be having the option to chose a specific folder as the source of all requests.
History and other metadata might not be interesting to share so they should be separated.
The UI should refresh when files change, like when you do a git pull on the CLI.
@liyasthomas commented on GitHub (Oct 30, 2020):
I'm seeking a GitHub app starter kit with the basic CRUD implementations on a user/organization repo.
Edit: closed the issue by mistake.
@gempir commented on GitHub (Oct 30, 2020):
Could be a good feature aswell but not at all what the issue creator was asking for I think. Imagine a corporate environment without any access to github.
@liyasthomas commented on GitHub (Oct 30, 2020):
Well, almost all platforms (BitBucket, GitLab etc) has Oauth apps. I mentioned GitHub as a starting point for this feature. There's always scope for other platforms as well.
@gempir commented on GitHub (Oct 30, 2020):
Yes but why go through all that trouble if storing files in plaintext in a specific folder could already solve most of the issues.
Let the user handle persisting and syncing. They could put it in Google drive or Dropbox if they wanted to. Or like we discussed a git repo.
All the app has to do is detect changes and update the UI accordingly
@liyasthomas commented on GitHub (Oct 30, 2020):
Got it. So whatever the platform of storage be (Google drive, Dropbox, Git repo etc), it's API access should allow us to sync data upon changes. A feature which sync collections irrespective of the choosen source.
@AndrewBastin wdys?
@al3rez commented on GitHub (Oct 30, 2020):
I think it's always better to do it first and make it perfect later.
@AndrewBastin commented on GitHub (Nov 1, 2020):
Well, I guess with the plans to update the store and global state system, we can figure out a way to emit a file.
But the actual challenge (Hoppscotch being a web app) is making and editing the files. The current web specs don't really implement a good solution for that. We can make the users download down a file, but not actually update and work with files in the background.
That is the case for local files.
For online (Git, Dropbox, Drive etc.) sync, it is pretty trivial. But we need to build up a good backend infrastructure to implement it properly. We are working on the backend infrastructure (for Teams, stay tuned for more info on that) and if that all goes on good, we could work on this down the road.
As far as I can think, these are the limitations and challenges on implementing a solution to this.
@liyasthomas commented on GitHub (Nov 1, 2020):
Well, i did a bg check on how apps like draw.io does this. They and lotta other web apps are making use of Native File API for local files. This works well in most browsers now.
https://web.dev/file-system-access
https://wicg.github.io/file-system-access
https://developer.mozilla.org/en-US/docs/Web/API/FileSystem
If there's a emited file, I guess incorporating File system solves saving it to local dir. At the same time, Google Drive, Git repo etc are another thing.
As Andrew mentioned, current priority is set to releasing Teams support asap. And after that we can definitely work on improving syncing logic.
@AndrewBastin commented on GitHub (Nov 1, 2020):
@liyasthomas, pretty sure you checked the compat info for the
FilesystemAPI. That is an old standard. But the new API is under an experiment flag for only Chrome.draw.io AFAIK just uses the techniques even we are using right now in Hoppscotch actually. Saving initiates a download. We can open from a file in the normal form input means.
Here is the browser compat details on the new API: https://caniuse.com/native-filesystem-api
@liyasthomas commented on GitHub (Nov 1, 2020):
I did, and we might need a legacy polyfill too, for implementing it as of now. 😁
@al3rez commented on GitHub (Nov 10, 2020):
https://github.com/char0n/http-request-in-editor
@liyasthomas commented on GitHub (Dec 9, 2020):
Users logged in with
GitHubcan now create secret gist of their Collections and Environments. Also allows importing both from a gist URL.@lanphan commented on GitHub (Mar 17, 2021):
@liyasthomas
I just download and try Hoppscotch, already created 2 collections + 3 environment set, found this and tried Github sync as your comment said (already login Github, authorized and enabled 3 sync: history, collection and environment), but I found some issues below:
@liyasthomas commented on GitHub (Mar 17, 2021):
Hi @lanphan creating gist is not an automatic process. We respect your privacy and you should manually create secret gist of collections.
This is exactly not the way this should work. We'll be automating this stuffs with sync feature. You can essentially toggle a switch from settings and everything works under the hood. This proof of concept will be refactored. Please be patient while we work on this feature. Meanwhile follow our public project board to track progress: https://github.com/hoppscotch/hoppscotch/projects/6
Thanks.
@andrestone commented on GitHub (Apr 1, 2021):
Isn't the original issue about having some sort of format to store collections and their state, so these can be subject to version control?
I think that having a standard / portable format would enable a lot of great things. It would be amazing to iterate using the GUI, then consume the requests defined in the collections programmatically on CI.
Maybe this is already supported and I can't find it?
@liyasthomas commented on GitHub (Apr 2, 2021):
We're working on a totally new state system (which is already implemented on some pages - #1560) and revamped collections protocol that would eventually solve this issue. Which allow us to version control collections and most importantly "Teams collaboration" with version control. And sync with gist or a JSON file was part of a proof of concept. Down the line as we work on https://github.com/hoppscotch/hoppscotch/projects/6, we'll addressing this issue and give prime focus.
Thanks for the suggestions, will let you know when this is resolved. Feel free to contribute to the project.
@andrestone commented on GitHub (Apr 2, 2021):
Thank you for the info.
It still feels like having a file protocol for collections is a separate issue. IMHO, synchronizing workspaces is a feature that might make use of the file protocol, but not necessarily.
@wouthoekstra commented on GitHub (Nov 16, 2021):
The solution that is being worked on (https://github.com/hoppscotch/hoppscotch/projects/6) suggests a single folder where all workspaces have to be stored in. Would it still allow for multiple projects to be stored in several places? For me the biggest advantage of having the collection be stored wherever I want would be that it would be possible to store the collection within the projects repository.
This would allow the collections status to be directly tied to the projects version. If I check out a branch in git (or vcs for that matter), where one of my colleagues created some new API endpoints, the collection could immediately reflect the new endpoints. If I check out the latest released branch, I would return to see just the endpoints that are released.
With one central workspace folder there is more mental load to store the collections for endpoints other than the latest released APIs, especially when you have to work on multiple projects.
@AndrewBastin commented on GitHub (Nov 16, 2021):
@wouthoekstra The Github Project boards we are using are really outdated and we have backtracked on the implementation plans discussed on that. We want to focus on a more robust implementation in which you can actually write your Hoppscotch collections as a file (kinda like an export, but not exactly) and work off on that so we don't exactly worry about the Git aspect of the things.
But before getting into this, we have two major blockers:
We wanted to actually our UX around collections and workspaces in a way multiple sources (like personal collections, team collections and (when this lands) file based collections) are cleanly expressed to the user. (cc: @liyasthomas)
To provide a good user experience for this feature while still being a web app, we want proper support for the File System Access API which hasn't landed yet on non-Chromium browsers (our compat policy for features requires support for Chrome, Firefox and Safari (atleast their latest stable versions)). To counteract this and other browser issues/restrictions, there is active effort internally to build a native app (meanwhile maintaining our performance standards) which allows us to bypass that wait, but that is also a work in progress and won't be done soon.
We will update this thread accordingly once this actually gets into proper progress.
@liyasthomas can you look into the project boards ?
@liyasthomas commented on GitHub (Nov 16, 2021):
I've deleted the existing project board, it has been stale for a very long time, and our roadmap has been changed quite a lot.
Once we have a proper UX of the way users switch collections context from personal to team/organization, syncing with multiple providers like git/file can be taken into consideration. Watch this space for updates.
@wpggles commented on GitHub (Apr 11, 2022):
Sometimes I'm working in LAN with my teams , so I'm very interested in this feature (like self-hosted authorization and storage) . Any progress on this one ? Have roadmap or plan list ?
@jalberto commented on GitHub (May 24, 2022):
Just to clarify, right now the only way to store/sync collections is to use Hopscotch sync service, that uses Firebase.
Is that right?
If so, how safe are sensitive data in the collections or environment?
As I understand this request the goal is to have a single place in the local system to store everything, and then is up to the user to use that location in dropbox, git or foo
@karelbilek commented on GitHub (Feb 3, 2023):
This would be a killer feature that would literally made me switch from Postman
@isaiahdahl commented on GitHub (Jul 27, 2023):
There still any life in this feature? I'm definitely interested.
@massimeddu-sj commented on GitHub (Sep 22, 2023):
This is also the single feature preventing my team to use this great tool.
@federicorosso1993 commented on GitHub (Oct 12, 2023):
this would be great and the only thing that, while I'm trying this app, is really missing from this.
Basically I think it as
I got
1 collection (my current project)->multiple rest call.
open one->modify it->save
It will automatically export to my local git repository so that I can push it to any git compatible repository.
Right now I can do it manually, but it is just not the same.
The main problem I think it is that right now there is no Desktop app to do that (so you cannot really export anything to my local folders automatically) so you need to support each cloud platform individually).
@Aeolun commented on GitHub (Nov 14, 2023):
There is now a desktop app to do this :)
@federicorosso1993 commented on GitHub (Nov 14, 2023):
I tried it but it crash on startup, not sure why...
@xzycn commented on GitHub (Nov 14, 2023):
It's a bug, you can watch #3524 .
@LaSylv commented on GitHub (Nov 16, 2023):
https://github.com/hoppscotch/hoppscotch/issues/870#issuecomment-1809394789
How to do this with the desktop app ?
Can't find anything :/
@1Mark commented on GitHub (Nov 22, 2023):
Let me clarify. It is a PWA. It can be installed via
However, that was a response to
Claiming that there is no desktop app - which may have been true then - not important.
TL;DR Desktop app or not, there is still no support for git integration
@Bigua commented on GitHub (Dec 10, 2023):
Since this seems that wil take some time, I've migrated to usebruno, they have support to save all colections in plain text on a folder who can be synced via git.
@1Mark commented on GitHub (Dec 10, 2023):
Great. Thanks for letting us know.
@karelbilek commented on GitHub (Dec 18, 2023):
( @Bigua do thank you, I didn't know about usebruno :P )
@lauhon commented on GitHub (Jan 18, 2024):
This is basically the only thing missing for me to pitch my employer to use hoppscotch for all of their API testing and documentation. Really would love to see this coming! :)
@Bigua - how can your usebruno integration be used, is there documentation somewhere?
@Bigua commented on GitHub (Jan 25, 2024):
Sadly the doc there its incipient , so you can do this tinkering with the app, I personally don't use it with integrations
@a-super-admin commented on GitHub (Feb 16, 2024):
Hi, Great tool.
The original issue poster (https://github.com/hoppscotch/hoppscotch/issues/1309#issue-733576400) from 4 years ago, and me and dozens of other fans of this app just wanted to have the persistance to be on a filesystem. Especially when you do self-hosting, wanting to store, sync (maybe sensitive) collections on our own disk, easily adoptable with a docker mount. I don't understand it yet, we can only persist collections on 3rd party cloud services/VCSs?
I just need the json blob that sits in the browser local storage to be read/written from/to disk, everything else like syncing, git push, pull etc is then up to the maintainer. Am I missing something?
Thank you!
@equiman commented on GitHub (Mar 15, 2024):
I would like to propose an MVP solution on this:
Would it be possible?
The responsibility will fall on the user whether or not to update these files. But also, it will give us the opportunity to use all the git benefits to use different like branching and versioning and the most important use a private repository to not expose sensitive information.
@nazimuddintbasha commented on GitHub (May 29, 2024):
Looks like lot of interest shown for the Git Sync feature but don't see any movement. I see few of them chipped in with the ideas. I really appreciate the community members striving for new features. Any thoughts on how we get this feature roll out?
@pavel-suhakou commented on GitHub (Jul 13, 2024):
They don’t want to do it, use Bruno instead.
@hemachandsai commented on GitHub (Aug 13, 2024):
Exactly bruno has this feature, i will try it
@ImNicolasTheDev commented on GitHub (Dec 4, 2024):
Hi there ! After more than 4 years, this is still a highly requested feature.
One solution could be, in the Options tab, to have multiple fields to configure Git syncing. This would allow, not only for Github repo sync, but also for private repos, Gitlab repos...
The only thing is to find a git client to be able to interact with it.
The process could be like so :
First, setup the git configuration in the Options (+ a ckeckbox should be available to automatically sync with git when Hoppscotch starts)
Then some buttons appear, like "Sync with Git" => clicking it will automatically pull and import the requests and environments json (it will only change the requests and envs with the same id, without deleting other things if the user have other requests for example.)
💡 An idea would be to make it like requests and envs stored in git have unique ids (like add
gitin the id :git1a2b3c4d). A problem that could appear is that if a new git repo containing requests is pulled, and locally the user have a request with the same id, the local request would be overwritten.Then, after you synced (pulled) the new repo, the button changes to "Push changes", which stays grey until you modify any request / variable. Let's say we modified a variable in a request. We can now click on "Push changes", which will first make a new window appear which contains :
(The git client will first pull changes if any, merge them if everything's fine, and then commit and push)
And now the user successfully pushed the modifications to Github, Gitlab... ! 😄
BUT, THERE ARE ANNOYING CASES :
First, if two users are modifying the same things (a specific variable, the url of a request, a part of the request's body, could be anything) at the same time. The 1st one pushes changes, no problem. But then, when the 2nd user tries to push the modifications, he encounter an error.
💡 Hoppscotch could display a window containing mutiple things :
Second annoying case is, if the user hasn't used Hoppscotch in a while and has the "automatically sync with git" option checked (so it's activated), and has unpushed modifications locally. When he will start Hoppscotch, and the pull action happens, there's going to be merge conflicts.
💡 A solution could be to reuse the "Merge conflicts" window !
That's all I could think about GIT integration in Hoppscotch!
@liyasthomas do you think it's something that could be prioritized since this feature is highly requested?
Oh, and all of the things mentioned above are just some ideas, feel free to give me some feedback, and to add some ideas too! This is how we can move the feature forward 😉
@bruno-lopes commented on GitHub (Jan 29, 2025):
This would be a really nice feature. After performing some changes, the user should inform Hoppscotch that it would like to push the changes. Hoppscotch should create a new branch, and automatically start a pull request / merge request to the main branch. When the merge occurs and this "feature branch" is destroyed, Hoppscotch identifies those changes and automatically updates local collection.
@jasine commented on GitHub (Feb 28, 2025):
+1
@deidei1216 commented on GitHub (Mar 21, 2025):
+1
@xeon-zolt commented on GitHub (Mar 26, 2025):
+1
@lamongabriel commented on GitHub (Apr 24, 2025):
+1
@dungmv commented on GitHub (Jun 4, 2025):
upvote
@cvionnet commented on GitHub (Jun 20, 2025):
+1
@ccxnu commented on GitHub (Jul 29, 2025):
+1
@ducvuongpham commented on GitHub (Aug 19, 2025):
+1
@weirdo-adam commented on GitHub (Aug 27, 2025):
+1
@samkellerhals commented on GitHub (Sep 12, 2025):
+1
@fschein commented on GitHub (Sep 16, 2025):
+1
@giancarlomsoares commented on GitHub (Sep 16, 2025):
+1
@pedrosimao commented on GitHub (Oct 1, 2025):
+1
@liudonghua123 commented on GitHub (Oct 30, 2025):
+1
@Maizi12 commented on GitHub (Dec 16, 2025):
+1
@maifeeulasad commented on GitHub (Dec 25, 2025):
We can do something like this:
let me know your opinion. And I am willing to work on this one, please assign this to me.
@martin-lysk commented on GitHub (Jan 22, 2026):
feat: git repository sync for collections #5797 tackles this, its still WIP so we (I am a co-author of the pr) would love to hear your thoughts