[PR #5662] [CLOSED] feat : Introduce iterations & data feed (CSV/JSON) to address #5324, #5163 #5310

New issue

Closed

opened 2026-03-17 02:46:05 +03:00 by kerem · 0 comments

kerem commented

2026-03-17 02:46:05 +03:00

Owner

📋 Pull Request Information

Original PR: https://github.com/hoppscotch/hoppscotch/pull/5662
Author: @Manohar-ford
Created: 12/4/2025
Status: ❌ Closed

Base: main ← Head: collection-runner

📝 Commits (2)

22487b3 feat: add support for iterations and validation in collection runner
cd26755 feat: add data-driven testing support with CSV/JSON file uploads and dataset management

📊 Changes

8 files changed (+613 additions, -73 deletions)

View changed files

📝 packages/hoppscotch-common/locales/en.json (+16 -0)
📝 packages/hoppscotch-common/src/components/http/test/RunnerModal.vue (+319 -7)
📝 packages/hoppscotch-common/src/components/importExport/Base.vue (+0 -2)
📝 packages/hoppscotch-common/src/components/share/Modal.vue (+0 -1)
📝 packages/hoppscotch-common/src/helpers/RequestRunner.ts (+43 -12)
➕ packages/hoppscotch-common/src/helpers/import-export/dataset/parser.ts (+66 -0)
📝 packages/hoppscotch-common/src/helpers/rest/document.ts (+7 -0)
📝 packages/hoppscotch-common/src/services/test-runner/test-runner.service.ts (+162 -51)

📄 Description

Closes #5324
Closes #5163

Description

This Pull Request introduces significant enhancements to the Hoppscotch Collection Runner, enabling users to execute collections with simple iterations and drive request parameters using external data files. This allows for robust data-driven testing and automation of repetitive tasks, directly addressing user demands for more powerful collection execution capabilities.

What's Changed

This PR implements two major features within the Collection Runner: Simple Iteration and Data Feed (CSV/JSON).

1. Collection Runner - Simple Iteration

A new input field has been added to the Collection Runner UI, allowing users to specify a fixed number of times a collection (or selected requests within it) should be executed.
The runner will now loop through the entire collection N times, providing a straightforward way to repeat tests or actions.
Each iteration will run with a fresh state, allowing for consistent testing across multiple runs.

2. Collection Runner - Data Feed (CSV/JSON)

Users can now upload external data files (CSV or JSON format) to drive variables within their requests during collection runs.
A new section in the Collection Runner UI allows for selecting and previewing the uploaded data file.
For each row in the data file (or each object in a JSON array), a new iteration of the collection is executed.
Data from the current row/object is automatically injected as environment variables for that specific iteration.
- Example: A CSV column username would be accessible as {{username}} in requests during that iteration.
This enables dynamic population of request URLs, headers, body, and query parameters, facilitating data-driven testing scenarios.
Robust parsing for both CSV and JSON formats is implemented, with clear feedback for malformed files.

Integration and UI/UX

The Collection Runner interface has been updated to include controls for both iteration count and data file upload.
The execution logic of the Collection Runner has been refactored to manage the iteration loop and variable injection from data feeds.
Improved feedback mechanisms during the run to indicate current iteration number and data context.

Notes to Reviewers

Motivation
This feature directly responds to the community's need for advanced testing capabilities, as highlighted in issues [feature]: Support setting number of iterations in Collection Runner UI #5324 and [feature]: Collection runner via GUI with ability to import files and choose which requests should be run #5163. It allows Hoppscotch to compete more effectively with other API clients in terms of automated testing and data-driven workflows.

Testing Strategy

Manual Testing: Extensive manual testing was performed to cover:
- Running collections with a specified number of iterations (e.g., 1, 5, 10).
- Uploading valid CSV and JSON files, ensuring correct variable injection into requests.
- Testing with various request types (GET, POST, PUT) and variable locations (URL, headers, body).
- Verifying error handling for malformed data files or missing variables.
- Confirming that environments are correctly handled per iteration.
Unit/Integration Tests: Relevant unit and integration tests have been added/updated for the new iteration and data parsing logic.

Edge Cases Considered

Empty data files: Handled gracefully, resulting in zero iterations or an appropriate error message.
Missing variables in data: Variables not found in the data file will fall back to existing environment/global variables or remain unresolved, consistent with current Hoppscotch behavior.
Large data files: Performance considerations were made for parsing, though extremely large files might still impact initial load time.
Data type handling: All data from CSV/JSON is treated as strings for variable substitution, allowing users to perform type conversions within their request scripts if needed.

Future Enhancements (Out of Scope for this PR)

Advanced reporting for data-driven runs (e.g., pass/fail statistics per iteration).
Support for additional data formats (e.g., Excel).
More sophisticated iteration control (e.g., running specific rows, conditional iterations).
Visual mapping of data file columns to request variables.

Backward Compatibility
This feature is fully backward compatible. Existing collection runs without iteration counts or data files will function as before.

Security Considerations
Data files are processed client-side. Users are responsible for the content of their uploaded files. No new external network requests are made as part of the data parsing.

Checklist

Linked issues (#5324, #5163)
Backward compatible
New UI elements for iteration count and data file upload
Collection Runner logic updated for iteration and data feed
CSV data parsing implemented
JSON data parsing implemented
Data variables injected into request context for each iteration

Summary by cubic

Adds iterations and CSV/JSON data feed to the Collection Runner to enable repeatable and data-driven runs. Closes #5324 and #5163.

New Features
- Iterations input with validation (must be ≥1); Run disabled when invalid.
- Upload CSV or JSON as a dataset; auto-parse, validate, and set iterations to row count.
- Inject dataset row values as temp environment variables per run (e.g., {{username}}).
- Data preview modal with download and clear actions.
- Runner executes N iterations with optional delay; stops on error if configured.
- Accumulates request results across iterations; folders initialized once.
- Passes iteration data into RequestRunner; resolves requests with enriched envs.
- Added locale strings for new UI labels and messages.

^{Written for commit cd26755a6f. Summary will update automatically on new commits.}

_{🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.}

## 📋 Pull Request Information **Original PR:** https://github.com/hoppscotch/hoppscotch/pull/5662 **Author:** [@Manohar-ford](https://github.com/Manohar-ford) **Created:** 12/4/2025 **Status:** ❌ Closed **Base:** `main` ← **Head:** `collection-runner` --- ### 📝 Commits (2) - [`22487b3`](https://github.com/hoppscotch/hoppscotch/commit/22487b3cd5f8dd86fd5423db76edf84871ef1141) feat: add support for iterations and validation in collection runner - [`cd26755`](https://github.com/hoppscotch/hoppscotch/commit/cd26755a6f0b9b7b5c0f52f5f53aa7e48bfa2b27) feat: add data-driven testing support with CSV/JSON file uploads and dataset management ### 📊 Changes **8 files changed** (+613 additions, -73 deletions) <details> <summary>View changed files</summary> 📝 `packages/hoppscotch-common/locales/en.json` (+16 -0) 📝 `packages/hoppscotch-common/src/components/http/test/RunnerModal.vue` (+319 -7) 📝 `packages/hoppscotch-common/src/components/importExport/Base.vue` (+0 -2) 📝 `packages/hoppscotch-common/src/components/share/Modal.vue` (+0 -1) 📝 `packages/hoppscotch-common/src/helpers/RequestRunner.ts` (+43 -12) ➕ `packages/hoppscotch-common/src/helpers/import-export/dataset/parser.ts` (+66 -0) 📝 `packages/hoppscotch-common/src/helpers/rest/document.ts` (+7 -0) 📝 `packages/hoppscotch-common/src/services/test-runner/test-runner.service.ts` (+162 -51) </details> ### 📄 Description <img width="767" height="782" alt="collection runner 1" src="https://github.com/user-attachments/assets/7236b135-cc9b-44fa-88ba-6a57bfcf9544" /> <img width="759" height="781" alt="collection runner" src="https://github.com/user-attachments/assets/345c99d3-1184-42a6-aac3-8a28b16a4254" /> <img width="760" height="699" alt="collection data feed" src="https://github.com/user-attachments/assets/e7cfae81-2531-4c57-8acd-3dab9100048a" /> Closes #5324 Closes #5163 ### Description This Pull Request introduces significant enhancements to the Hoppscotch Collection Runner, enabling users to execute collections with simple iterations and drive request parameters using external data files. This allows for robust data-driven testing and automation of repetitive tasks, directly addressing user demands for more powerful collection execution capabilities. ### What's Changed This PR implements two major features within the Collection Runner: **Simple Iteration** and **Data Feed (CSV/JSON)**. #### 1. Collection Runner - Simple Iteration - A new input field has been added to the Collection Runner UI, allowing users to specify a fixed number of times a collection (or selected requests within it) should be executed. - The runner will now loop through the entire collection `N` times, providing a straightforward way to repeat tests or actions. - Each iteration will run with a fresh state, allowing for consistent testing across multiple runs. #### 2. Collection Runner - Data Feed (CSV/JSON) - Users can now upload external data files (**CSV** or **JSON** format) to drive variables within their requests during collection runs. - A new section in the Collection Runner UI allows for selecting and previewing the uploaded data file. - For each row in the data file (or each object in a JSON array), a new iteration of the collection is executed. - Data from the current row/object is automatically injected as environment variables for that specific iteration. - *Example:* A CSV column `username` would be accessible as `{{username}}` in requests during that iteration. - This enables dynamic population of request URLs, headers, body, and query parameters, facilitating data-driven testing scenarios. - Robust parsing for both CSV and JSON formats is implemented, with clear feedback for malformed files. #### Integration and UI/UX - The Collection Runner interface has been updated to include controls for both iteration count and data file upload. - The execution logic of the Collection Runner has been refactored to manage the iteration loop and variable injection from data feeds. - Improved feedback mechanisms during the run to indicate current iteration number and data context. --- ### Notes to Reviewers **Motivation** This feature directly responds to the community's need for advanced testing capabilities, as highlighted in issues [feature]: Support setting number of iterations in Collection Runner UI #5324 and [feature]: Collection runner via GUI with ability to import files and choose which requests should be run #5163. It allows Hoppscotch to compete more effectively with other API clients in terms of automated testing and data-driven workflows. **Testing Strategy** * **Manual Testing:** Extensive manual testing was performed to cover: * Running collections with a specified number of iterations (e.g., 1, 5, 10). * Uploading valid CSV and JSON files, ensuring correct variable injection into requests. * Testing with various request types (GET, POST, PUT) and variable locations (URL, headers, body). * Verifying error handling for malformed data files or missing variables. * Confirming that environments are correctly handled per iteration. * **Unit/Integration Tests:** Relevant unit and integration tests have been added/updated for the new iteration and data parsing logic. **Edge Cases Considered** * **Empty data files:** Handled gracefully, resulting in zero iterations or an appropriate error message. * **Missing variables in data:** Variables not found in the data file will fall back to existing environment/global variables or remain unresolved, consistent with current Hoppscotch behavior. * **Large data files:** Performance considerations were made for parsing, though extremely large files might still impact initial load time. * **Data type handling:** All data from CSV/JSON is treated as strings for variable substitution, allowing users to perform type conversions within their request scripts if needed. **Future Enhancements (Out of Scope for this PR)** * Advanced reporting for data-driven runs (e.g., pass/fail statistics per iteration). * Support for additional data formats (e.g., Excel). * More sophisticated iteration control (e.g., running specific rows, conditional iterations). * Visual mapping of data file columns to request variables. **Backward Compatibility** This feature is fully backward compatible. Existing collection runs without iteration counts or data files will function as before. **Security Considerations** Data files are processed client-side. Users are responsible for the content of their uploaded files. No new external network requests are made as part of the data parsing. --- ### Checklist - [x] Linked issues (#5324, #5163) - [x] Backward compatible - [x] New UI elements for iteration count and data file upload - [x] Collection Runner logic updated for iteration and data feed - [x] CSV data parsing implemented - [x] JSON data parsing implemented - [x] Data variables injected into request context for each iteration  --- ## Summary by cubic Adds iterations and CSV/JSON data feed to the Collection Runner to enable repeatable and data-driven runs. Closes #5324 and #5163. - **New Features** - Iterations input with validation (must be ≥1); Run disabled when invalid. - Upload CSV or JSON as a dataset; auto-parse, validate, and set iterations to row count. - Inject dataset row values as temp environment variables per run (e.g., {{username}}). - Data preview modal with download and clear actions. - Runner executes N iterations with optional delay; stops on error if configured. - Accumulates request results across iterations; folders initialized once. - Passes iteration data into RequestRunner; resolves requests with enriched envs. - Added locale strings for new UI labels and messages. <sup>Written for commit cd26755a6f0b9b7b5c0f52f5f53aa7e48bfa2b27. Summary will update automatically on new commits.</sup>  --- <sub>🔄 This issue represents a GitHub Pull Request. It cannot be merged through Gitea due to API limitations.</sub>