[GH-ISSUE #71] Test exhaustively by creating test input files from SDK model #145

New issue

Open

opened 2026-03-15 11:47:15 +03:00 by kerem · 1 comment

kerem commented

2026-03-15 11:47:15 +03:00

Owner

Originally created by @mschlaipfer on GitHub (Dec 11, 2025).
Original GitHub issue: https://github.com/awslabs/iam-policy-autopilot/issues/71

To test extraction exhaustively we should generate input files from the SDK model underlying the extractor to check that we extract all methods.

I.e., for all operations: get the operation in the botocore model, generate a Python, Go, etc. file that sets up the call, including all required inputs, and import statements (if applicable depending on the language), and checks that after extraction we have the expected operation in the SdkMethodCall.

As an improvement, we can look up the AuthorizedActions in the service reference for the operation we are testing to check that the final policy contains all the expected actions given an initial operation. If we had had this, this would have prevented https://github.com/awslabs/iam-policy-autopilot/issues/66

Originally created by @mschlaipfer on GitHub (Dec 11, 2025). Original GitHub issue: https://github.com/awslabs/iam-policy-autopilot/issues/71 To test extraction exhaustively we should generate input files from the SDK model underlying the extractor to check that we extract all methods. I.e., for all operations: get the operation in the botocore model, generate a Python, Go, etc. file that sets up the call, including all required inputs, and import statements (if applicable depending on the language), and checks that after extraction we have the expected operation in the `SdkMethodCall`. As an improvement, we can look up the `AuthorizedActions` in the service reference for the operation we are testing to check that the final policy contains all the expected actions given an initial operation. If we had had this, this would have prevented https://github.com/awslabs/iam-policy-autopilot/issues/66

kerem added the

code-quality

label

2026-03-15 11:47:15 +03:00

kerem commented

2026-03-15 11:47:26 +03:00

Author

Owner

@mschlaipfer commented on GitHub (Dec 11, 2025):

Even a simpler version, just starting from enrichment using the method names (i.e., snake_case(SdkModelOperationName) for Python and SdkModelOperationName for other languages) and checking that the policy has all authorized actions would have been sufficient to prevent https://github.com/awslabs/iam-policy-autopilot/issues/66.

@mschlaipfer commented on GitHub (Dec 11, 2025): Even a simpler version, just starting from enrichment using the method names (i.e., `snake_case(SdkModelOperationName)` for Python and `SdkModelOperationName` for other languages) and checking that the policy has all authorized actions would have been sufficient to prevent https://github.com/awslabs/iam-policy-autopilot/issues/66.

kerem referenced this issue

2026-03-15 11:56:24 +03:00

[PR #145] [CLOSED] feat(mcp): enable policy explanations in generate_policy tool #264