[PR #3425] [MERGED] refactor: revamp the importers & exporters systems to be reused #4379

New issue

Closed

opened 2026-03-17 01:55:12 +03:00 by kerem · 0 comments

kerem commented

2026-03-17 01:55:12 +03:00

Owner

📋 Pull Request Information

Original PR: https://github.com/hoppscotch/hoppscotch/pull/3425
Author: @amk-dev
Created: 10/5/2023
Status: ✅ Merged
Merged: 12/6/2023
Merged by: @AndrewBastin

Base: release/2023.12.0 ← Head: feat/env-importers

📝 Commits (10+)

adcf478 refactor: add import sources and components
291602f chore: make base import export components
3f2aaf7 refactor: extract importing from my collections ui
69a6061 refactor: refactor collections import export
82464ce refactor: refactor gql collections import export
e6358b3 refactor: refactor environments import export
656dd42 chore: add i18n strings
ddf2b2f refactor: add some missing exporters/importers
d737f29 refactor: refactor import from url using query params to use the new system
19ef6de chore: remove old code no longer used

📊 Changes

90 files changed (+2356 additions, -1849 deletions)

View changed files

📝 packages/hoppscotch-common/.eslintrc.js (+2 -0)
📝 packages/hoppscotch-common/locales/en.json (+14 -2)
📝 packages/hoppscotch-common/src/components/app/Inspection.vue (+1 -2)
📝 packages/hoppscotch-common/src/components/collections/Collection.vue (+3 -4)
📝 packages/hoppscotch-common/src/components/collections/ImportExport.vue (+519 -312)
📝 packages/hoppscotch-common/src/components/collections/MyCollections.vue (+9 -11)
📝 packages/hoppscotch-common/src/components/collections/Request.vue (+1 -2)
📝 packages/hoppscotch-common/src/components/collections/SaveRequest.vue (+15 -17)
📝 packages/hoppscotch-common/src/components/collections/TeamCollections.vue (+71 -76)
📝 packages/hoppscotch-common/src/components/collections/graphql/Collection.vue (+1 -1)
📝 packages/hoppscotch-common/src/components/collections/graphql/Folder.vue (+1 -1)
📝 packages/hoppscotch-common/src/components/collections/graphql/ImportExport.vue (+175 -247)
📝 packages/hoppscotch-common/src/components/collections/graphql/index.vue (+1 -1)
📝 packages/hoppscotch-common/src/components/collections/index.vue (+54 -187)
📝 packages/hoppscotch-common/src/components/environments/Add.vue (+8 -9)
📝 packages/hoppscotch-common/src/components/environments/ImportExport.vue (+235 -346)
📝 packages/hoppscotch-common/src/components/environments/Selector.vue (+36 -43)
📝 packages/hoppscotch-common/src/components/environments/my/Details.vue (+5 -7)
📝 packages/hoppscotch-common/src/components/environments/my/index.vue (+1 -1)
📝 packages/hoppscotch-common/src/components/environments/teams/Details.vue (+7 -11)

...and 70 more files

📄 Description

Closes HFE-237,HFE-204,HFE-217
Fixs #3303

Changes

revamp the importers and exporters systems to be reused

Previously, our import logic was shattered around a lot of places, and duplicated a lot of code. this PR adds a simple consistant way to add importers without leaking the implementation to different places.

Here are some details to help navigate the PR better.

We remove duplicating code by using Import Sources

Import sources are the different ways we can import collections,environments etc.

For example, we can import from a file, from a url etc. the flow will look like this,

FileSource/UrlSource/GistSource gets the data from the file/url/gist -> and that data is passed to hoppscotchImporter/insomniaImporter/openAPIImporter etc.

Importers can say which sources they support, and our UI will render those options for the user to select.

Previously we were duplicating logic to import from a file / url / gist etc across different files. but now we have Import Sources abstractions that prevents duplication.
- FileSource
  - a function that creates a step with the FileImport.vue component. this component renders a file input and emits the file content as a string. the logic to get the file content is also moved to this component. previously we were duplicating this logic across different places.
- UrlSource
  - a function that creates a step with the UrlImport.vue component. this component renders a url input and emits the url content as a string. the logic to get the url content is also moved to this component. previously we were duplicating this logic across different places.
- GistSource
  - This step uses the same component as UrlSource. but it has logic to fetch the gist content using github api instead of a just fetching the url. previously we were duplicating this logic across different places.
Making Step By Step UI more flexible and maintainable with useSteps composable.

Previously imports had 2 steps. one to select the importer and then to Select File/Url etc. this was implemented very imperatively/hardcoded with if statements. but now our requirements are more complex. for example, we need to support multiple sources for a single importer, which adds the need for 3 steps. doing it using the previous implementation was not an ideal solution. so we added a useSteps composable that allows us to define steps and navigate between them, easily.

Previously, we were duplicating the step by step UI logic across different places. eg: components/importexport/ImportExport.vue and components/collections/ImportExport.vue had the same logic. but they were duplicated. this PR adds a defineStep function that allows us to define a step and reuse it across different places. this makes the code more maintainable and flexible.

a new function defineStep is used to create a step that can be used in the step ui. a step is nothing but an component + its props, which can be rendered when needed. for example a simple flow looks like this.
- Step 1 - component: ImportExportList, props: { importers, exporters }, renders a list of importers and exporters
- Step 2 - component: ImportExportSourcesList, props: { sources }, renders a list of sources for the selected importer/exporter
- Step 3 - component: FileImport, props: { acceptedFileTypes, caption, onImportFromFile }, renders a file input and emits the file content as a string. the selected importer will be executed with the file content.
defineStep function takes an ID, the component to render for the Step. and its props/emits as parameters and it will create a step for you. the props/emits also gets nice typescript autocomplete as you would expect.
Extract duplicated UI logic to components/importexport/ImporterExport.vue
- We have 3 ImportExport UIs, all of them had the same logic. but they were duplicated and had inconsistant UIs. this PR adds a components/importexport/ImportExport.vue component, that act as the base component with all the common UI, it takes importerModules and exporterModules as props and renders the UI, also implements the step logic and handles navigating between the steps.

How Importers for Collections, Environments are implemented using the common ImportExport component

We create a simple wrapper for the common ImporterExport.vue component.
This wrapper defines the importerModules and exporterModules. and passes it to the components/importexport/ImporterExport.vue component.
The importers and exporters will have type ImporterOrExporter,
- See ImporterOrExporter type.
  - It has supported_sources for adding an array of supported sources. eg: FileSource, UrlSource ( eg: OpenAPI Importer supports UrlImport + File Import )
  - It has component for adding a single source/any component. eg: FileSource ( eg: All other importers other than OpenAPI uses FileSource only )
  - It has action for adding a function to execute without showing any UI. ( eg: Export as JSON uses this ).
- When the user selects an importer, if it has a component, we show the component UI, else we render the list of sources. it is also possible to just execute a function without showing any UI using action. ( eg: export as json use this.)

Here is an example importer defined,

const HoppEnvironmentsImport: ImporterOrExporter = {
  metadata: {
    // defines the metadata
  },
  // give the source
  component: FileSource({
    acceptedFileTypes: "application/json",
    caption: "import.hoppscotch_environment_description",
    onImportFromFile: async (environments) => {
      // executes the importer function
      // you can import importer functions from '~/helpers/import', they are just regular functions. Since we are using them directly instead of in a loop of importer functions, they don't have to follow any specific type. this adds a lot of flexibility + speed of iteration. but takes away a little convenience.
      const res = await hoppEnvImporter(environments)();

      // do component specific stuff
      if (E.isLeft(res)) {
        failedImport();
        return;
      } else {
        importSuccessful(res.right);
      }

      emit("hide-modal");
    },
  }),
};

Use the same system for collections, envs, graphql import&exports

Previously we had seperate components with inconsistant UIs for collections, envs, and gql imports/exports. since we use the same base components, now we have one consistant import/export UI all across the product.
Implement hoppEnv + postmanEnv imports with zod validation

Previously we didn't have a seperate entry for importing hopp environment and postman environment. but the code was kinda handling both of these platforms. this PR adds seperate entries in the Import/Export list. also these importers have zod validation. ( the schema for postman is created from the previous implementation. this can be improved in the future. )
OpenAPI imports from URL + File

Previously we only had support for importing openAPI from a file. but now we have support for importing from a url too. ( note: this can be enabled for any of our importers, but we only enabled it for openAPI for now. )

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/hoppscotch/hoppscotch/pull/3425 **Author:** [@amk-dev](https://github.com/amk-dev) **Created:** 10/5/2023 **Status:** ✅ Merged **Merged:** 12/6/2023 **Merged by:** [@AndrewBastin](https://github.com/AndrewBastin) **Base:** `release/2023.12.0` ← **Head:** `feat/env-importers` --- ### 📝 Commits (10+) - [`adcf478`](https://github.com/hoppscotch/hoppscotch/commit/adcf478164255747f547512cb1419fcc9d4f13d3) refactor: add import sources and components - [`291602f`](https://github.com/hoppscotch/hoppscotch/commit/291602fcc4d9c0c1cb1899c053397f11925b91a6) chore: make base import export components - [`3f2aaf7`](https://github.com/hoppscotch/hoppscotch/commit/3f2aaf74cc7d142927bf5c1820ce5e73225dceae) refactor: extract importing from my collections ui - [`69a6061`](https://github.com/hoppscotch/hoppscotch/commit/69a60616a1f741b26de04c946271d97f52a05430) refactor: refactor collections import export - [`82464ce`](https://github.com/hoppscotch/hoppscotch/commit/82464ce8ea8beed6b9ac587e836ffb711604f950) refactor: refactor gql collections import export - [`e6358b3`](https://github.com/hoppscotch/hoppscotch/commit/e6358b33910eee8356c9e5c010ce6aa382bc2f55) refactor: refactor environments import export - [`656dd42`](https://github.com/hoppscotch/hoppscotch/commit/656dd42528cc270fe6c9d66d61867a6d11376463) chore: add i18n strings - [`ddf2b2f`](https://github.com/hoppscotch/hoppscotch/commit/ddf2b2fd7bbeeb8c7beedec0e073ab970cbf1c32) refactor: add some missing exporters/importers - [`d737f29`](https://github.com/hoppscotch/hoppscotch/commit/d737f296b1913008fcaad8f214edb018b4f3704e) refactor: refactor import from url using query params to use the new system - [`19ef6de`](https://github.com/hoppscotch/hoppscotch/commit/19ef6de6d814327466dd87aa24678b18e97f4a44) chore: remove old code no longer used ### 📊 Changes **90 files changed** (+2356 additions, -1849 deletions) <details> <summary>View changed files</summary> 📝 `packages/hoppscotch-common/.eslintrc.js` (+2 -0) 📝 `packages/hoppscotch-common/locales/en.json` (+14 -2) 📝 `packages/hoppscotch-common/src/components/app/Inspection.vue` (+1 -2) 📝 `packages/hoppscotch-common/src/components/collections/Collection.vue` (+3 -4) 📝 `packages/hoppscotch-common/src/components/collections/ImportExport.vue` (+519 -312) 📝 `packages/hoppscotch-common/src/components/collections/MyCollections.vue` (+9 -11) 📝 `packages/hoppscotch-common/src/components/collections/Request.vue` (+1 -2) 📝 `packages/hoppscotch-common/src/components/collections/SaveRequest.vue` (+15 -17) 📝 `packages/hoppscotch-common/src/components/collections/TeamCollections.vue` (+71 -76) 📝 `packages/hoppscotch-common/src/components/collections/graphql/Collection.vue` (+1 -1) 📝 `packages/hoppscotch-common/src/components/collections/graphql/Folder.vue` (+1 -1) 📝 `packages/hoppscotch-common/src/components/collections/graphql/ImportExport.vue` (+175 -247) 📝 `packages/hoppscotch-common/src/components/collections/graphql/index.vue` (+1 -1) 📝 `packages/hoppscotch-common/src/components/collections/index.vue` (+54 -187) 📝 `packages/hoppscotch-common/src/components/environments/Add.vue` (+8 -9) 📝 `packages/hoppscotch-common/src/components/environments/ImportExport.vue` (+235 -346) 📝 `packages/hoppscotch-common/src/components/environments/Selector.vue` (+36 -43) 📝 `packages/hoppscotch-common/src/components/environments/my/Details.vue` (+5 -7) 📝 `packages/hoppscotch-common/src/components/environments/my/index.vue` (+1 -1) 📝 `packages/hoppscotch-common/src/components/environments/teams/Details.vue` (+7 -11) _...and 70 more files_ </details> ### 📄 Description Closes HFE-237,HFE-204,HFE-217 Fixs #3303 **Changes** 1. ### **revamp the importers and exporters systems to be reused** Previously, our import logic was shattered around a lot of places, and duplicated a lot of code. this PR adds a simple consistant way to add importers without leaking the implementation to different places. Here are some details to help navigate the PR better. - We remove duplicating code by using `Import Sources` Import sources are the different ways we can import collections,environments etc. For example, we can import from a file, from a url etc. the flow will look like this, `FileSource/UrlSource/GistSource gets the data from the file/url/gist` -> `and that data is passed to hoppscotchImporter/insomniaImporter/openAPIImporter` etc. Importers can say which sources they support, and our UI will render those options for the user to select. Previously we were duplicating logic to import from a file / url / gist etc across different files. but now we have Import Sources abstractions that prevents duplication. - `FileSource` - a function that creates a step with the FileImport.vue component. this component renders a file input and emits the file content as a string. the logic to get the file content is also moved to this component. previously we were duplicating this logic across different places. - `UrlSource` - a function that creates a step with the UrlImport.vue component. this component renders a url input and emits the url content as a string. the logic to get the url content is also moved to this component. previously we were duplicating this logic across different places. - `GistSource` - This step uses the same component as UrlSource. but it has logic to fetch the gist content using github api instead of a just fetching the url. previously we were duplicating this logic across different places. - Making Step By Step UI more flexible and maintainable with `useSteps` composable. Previously imports had 2 steps. one to select the importer and then to Select File/Url etc. this was implemented very imperatively/hardcoded with if statements. but now our requirements are more complex. for example, we need to support multiple sources for a single importer, which adds the need for 3 steps. doing it using the previous implementation was not an ideal solution. so we added a `useSteps` composable that allows us to define steps and navigate between them, easily. Previously, we were duplicating the step by step UI logic across different places. eg: `components/importexport/ImportExport.vue` and `components/collections/ImportExport.vue` had the same logic. but they were duplicated. this PR adds a `defineStep` function that allows us to define a step and reuse it across different places. this makes the code more maintainable and flexible. a new function `defineStep` is used to create a step that can be used in the step ui. a step is nothing but an component + its props, which can be rendered when needed. for example a simple flow looks like this. - Step 1 - `component: ImportExportList, props: { importers, exporters }`, renders a list of importers and exporters - Step 2 - `component: ImportExportSourcesList, props: { sources }`, renders a list of sources for the selected importer/exporter - Step 3 - `component: FileImport, props: { acceptedFileTypes, caption, onImportFromFile }`, renders a file input and emits the file content as a string. the selected importer will be executed with the file content. `defineStep` function takes an ID, the component to render for the Step. and its props/emits as parameters and it will create a step for you. the props/emits also gets nice typescript autocomplete as you would expect. - Extract duplicated UI logic to `components/importexport/ImporterExport.vue` - We have 3 ImportExport UIs, all of them had the same logic. but they were duplicated and had inconsistant UIs. this PR adds a `components/importexport/ImportExport.vue` component, that act as the base component with all the common UI, it takes `importerModules` and `exporterModules` as props and renders the UI, also implements the step logic and handles navigating between the steps. - How Importers for Collections, Environments are implemented using the common ImportExport component - We create a simple wrapper for the common `ImporterExport.vue` component. - This wrapper defines the importerModules and exporterModules. and passes it to the `components/importexport/ImporterExport.vue` component. - The importers and exporters will have type `ImporterOrExporter`, - See `ImporterOrExporter` type. - It has `supported_sources` for adding an array of supported sources. eg: FileSource, UrlSource ( eg: OpenAPI Importer supports UrlImport + File Import ) - It has `component` for adding a single source/any component. eg: FileSource ( eg: All other importers other than OpenAPI uses FileSource only ) - It has `action` for adding a function to execute without showing any UI. ( eg: Export as JSON uses this ). - When the user selects an importer, if it has a `component`, we show the component UI, else we render the list of sources. it is also possible to just execute a function without showing any UI using `action`. ( eg: export as json use this.) - Here is an example importer defined, - ```typescript const HoppEnvironmentsImport: ImporterOrExporter = { metadata: { // defines the metadata }, // give the source component: FileSource({ acceptedFileTypes: "application/json", caption: "import.hoppscotch_environment_description", onImportFromFile: async (environments) => { // executes the importer function // you can import importer functions from '~/helpers/import', they are just regular functions. Since we are using them directly instead of in a loop of importer functions, they don't have to follow any specific type. this adds a lot of flexibility + speed of iteration. but takes away a little convenience. const res = await hoppEnvImporter(environments)(); // do component specific stuff if (E.isLeft(res)) { failedImport(); return; } else { importSuccessful(res.right); } emit("hide-modal"); }, }), }; ``` 2. ### **Use the same system for collections, envs, graphql import&exports** Previously we had seperate components with inconsistant UIs for collections, envs, and gql imports/exports. since we use the same base components, now we have one consistant import/export UI all across the product. 3. ### **Implement `hoppEnv` + `postmanEnv` imports with zod validation** Previously we didn't have a seperate entry for importing hopp environment and postman environment. but the code was kinda handling both of these platforms. this PR adds seperate entries in the Import/Export list. also these importers have zod validation. ( the schema for postman is created from the previous implementation. this can be improved in the future. ) 4. ### **OpenAPI imports from URL + File** Previously we only had support for importing openAPI from a file. but now we have support for importing from a url too. ( note: this can be enabled for any of our importers, but we only enabled it for openAPI for now. ) --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>