[PR #184] ADD e2e test step #310

New issue

Closed

opened 2026-03-02 04:13:15 +03:00 by kerem · 0 comments

kerem commented

2026-03-02 04:13:15 +03:00

Owner

Original Pull Request: https://github.com/git-ai-project/git-ai/pull/184

State: closed
Merged: Yes

Add End-to-End Test Suite

Motivation

During local testing, several issues and edge cases were discovered that weren't caught by unit tests alone. This PR introduces a comprehensive end-to-end test suite to prevent regression and ensure git-ai works correctly in real-world user scenarios.

Changes

This PR adds:

E2E Test Suite (tests/e2e/user-scenarios.bats) - 1,452 lines
- 15 comprehensive test scenarios covering real user workflows
- BATS-based testing framework for shell/git integration testing
- Helper functions for JSON validation and stats comparison
- Automated setup/teardown of test repositories
GitHub Actions Workflow (.github/workflows/e2e-tests.yml)
- Runs automatically on PRs and pushes to main
- Tests on Ubuntu with full Rust build pipeline
- Includes all dependencies (BATS, jq) for complete testing

Test Coverage

The test suite covers the following scenarios:

✅ Basic workflows: User and AI collaborative code creation
✅ Checkpoint functionality: Mock AI integration and file-specific checkpoints
✅ Attribution tracking: git-ai blame and git-ai stats accuracy
✅ Code modifications: AI/human deletions, refactoring, empty lines handling
✅ File operations: New file creation by AI
✅ Advanced Git operations:
- Squash authorship concatenation
- Rebase with AI authorship preservation
- Conflict resolution during rebase
- Reset and recommit scenarios

Detailed Test Cases

basic workflow: user creates file, AI adds code, user adds more code
git-ai checkpoint exits successfully without errors
git-ai checkpoint mock_ai with file path
git-ai blame shows correct attribution after commit
multiple checkpoints in sequence
git-ai stats shows AI contribution after commit
AI deletes lines from file
Human deletes lines from AI-generated code
AI generates code with empty lines in between
AI creates a new file from scratch
squash-authorship should concatenate AI and human changes
AI refactors its own code - squash-authorship should show no ai_deletions
Two AI commits, reset last commit, then recommit
AI authorship is preserved after rebase
AI attribution is preserved after fixing conflict during rebase

Benefits

🛡️ Prevents regressions - Catches issues before they reach production
🔍 Real-world testing - Tests actual user workflows, not just isolated functions
🤖 Automated CI - Runs on every PR automatically
📊 Better coverage - Validates end-to-end integration of git operations with AI tracking

Testing

All 15 test cases pass successfully:

bats tests/e2e/user-scenarios.bats

CI Integration

The E2E tests run automatically via GitHub Actions on:

All pull requests targeting main
All pushes to main

The workflow builds git-ai from source, installs all dependencies, and runs the complete test suite in a clean Ubuntu environment.

Note

Introduces a BATS-based E2E test suite and GitHub Actions workflow, with Taskfile targets to run tests in debug/release builds.

E2E Testing:

Add tests/e2e/user-scenarios.bats with helper functions and comprehensive scenarios (checkpointing, blame/stats validation, deletions, new files, empty lines, rebase/squash/reset flows; some marked skip).

CI:

New workflow /.github/workflows/e2e-tests.yml to build (Rust), install deps (bats, jq, task), and run task test:e2e on PRs and pushes to main.

Taskfile:

Add test:e2e and test:e2e:release tasks (depend on build-debug/build) to run the BATS suite.

^{Written by Cursor Bugbot for commit d5d5140502. This will update automatically on new commits. Configure here.}

**Original Pull Request:** https://github.com/git-ai-project/git-ai/pull/184 **State:** closed **Merged:** Yes --- # Add End-to-End Test Suite ## Motivation During local testing, several issues and edge cases were discovered that weren't caught by unit tests alone. This PR introduces a comprehensive end-to-end test suite to prevent regression and ensure `git-ai` works correctly in real-world user scenarios. ## Changes This PR adds: 1. **E2E Test Suite** (`tests/e2e/user-scenarios.bats`) - 1,452 lines - 15 comprehensive test scenarios covering real user workflows - BATS-based testing framework for shell/git integration testing - Helper functions for JSON validation and stats comparison - Automated setup/teardown of test repositories 2. **GitHub Actions Workflow** (`.github/workflows/e2e-tests.yml`) - Runs automatically on PRs and pushes to `main` - Tests on Ubuntu with full Rust build pipeline - Includes all dependencies (BATS, jq) for complete testing ## Test Coverage The test suite covers the following scenarios: - ✅ **Basic workflows**: User and AI collaborative code creation - ✅ **Checkpoint functionality**: Mock AI integration and file-specific checkpoints - ✅ **Attribution tracking**: `git-ai blame` and `git-ai stats` accuracy - ✅ **Code modifications**: AI/human deletions, refactoring, empty lines handling - ✅ **File operations**: New file creation by AI - ✅ **Advanced Git operations**: - Squash authorship concatenation - Rebase with AI authorship preservation - Conflict resolution during rebase - Reset and recommit scenarios ### Detailed Test Cases 1. `basic workflow: user creates file, AI adds code, user adds more code` 2. `git-ai checkpoint exits successfully without errors` 3. `git-ai checkpoint mock_ai with file path` 4. `git-ai blame shows correct attribution after commit` 5. `multiple checkpoints in sequence` 6. `git-ai stats shows AI contribution after commit` 7. `AI deletes lines from file` 8. `Human deletes lines from AI-generated code` 9. `AI generates code with empty lines in between` 10. `AI creates a new file from scratch` 11. `squash-authorship should concatenate AI and human changes` 12. `AI refactors its own code - squash-authorship should show no ai_deletions` 13. `Two AI commits, reset last commit, then recommit` 14. `AI authorship is preserved after rebase` 15. `AI attribution is preserved after fixing conflict during rebase` ## Benefits - 🛡️ **Prevents regressions** - Catches issues before they reach production - 🔍 **Real-world testing** - Tests actual user workflows, not just isolated functions - 🤖 **Automated CI** - Runs on every PR automatically - 📊 **Better coverage** - Validates end-to-end integration of git operations with AI tracking ## Testing All 15 test cases pass successfully: ```bash bats tests/e2e/user-scenarios.bats ``` ## CI Integration The E2E tests run automatically via GitHub Actions on: - All pull requests targeting `main` - All pushes to `main` The workflow builds `git-ai` from source, installs all dependencies, and runs the complete test suite in a clean Ubuntu environment.  --- > [!NOTE] > Introduces a BATS-based E2E test suite and GitHub Actions workflow, with Taskfile targets to run tests in debug/release builds. > > - **E2E Testing**: > - Add `tests/e2e/user-scenarios.bats` with helper functions and comprehensive scenarios (checkpointing, blame/stats validation, deletions, new files, empty lines, rebase/squash/reset flows; some marked `skip`). > - **CI**: > - New workflow `/.github/workflows/e2e-tests.yml` to build (Rust), install deps (`bats`, `jq`, `task`), and run `task test:e2e` on PRs and pushes to `main`. > - **Taskfile**: > - Add `test:e2e` and `test:e2e:release` tasks (depend on `build-debug`/`build`) to run the BATS suite. > > <sup>Written by [Cursor Bugbot](https://cursor.com/dashboard?tab=bugbot) for commit d5d514050259d5e0276613572069762a4baafa33. This will update automatically on new commits. Configure [here](https://cursor.com/dashboard?tab=bugbot).</sup>