Generating Tests with AI: Tips for High-Quality Test Design

Prev Next

You can generate tests with AI to transform high-level requirements into structured, business-process-oriented test cases. This article explains how to use the free text option best to generate tests that align perfectly with your organization’s internal standards, naming conventions, and reporting needs.

The “Generate tests with AI” wizard acts as your enterprise risk-based test architect and processes your input through three layers, as listed in this article.

  • Business Logic Layer
    The business logic layer identifies business milestones rather than UI actions.
    For example, "Create Purchase Order" and "Post Goods Receipt".

  • System Awareness

    For SAP → Panaya assigns relevant T-Codes.

    Non-SAP systems → the focus is on the process (not technical IDs).

  • Constraint Engine

    Applies your Additional Instructions to control -

    • Naming

    • Scope

    • Prioritization

    • Scenario types

Prompts Examples

  1. SAP Order-to-Cash

    Scenario - The requirement is for a new standard sales process, but it must ensure it covers specific financial blocks.

    Input Example - "Generate a suite focusing on the standard OTC flow. Include a specific negative scenario for a customer with a 'Credit Block' and a data validation scenario for 'Zero-Value' line items. Ensure the SAP T-Codes VA01, VL01N, and VF01 are mapped to their respective milestones."

  2. Cross-Functional Integration (SAP & Salesforce)

    Scenario- A process that starts in a CRM and flows into an ERP.

    Input Example - "Create a test that starts with 'Opportunity Closure' in Salesforce and ends with 'Invoice Generation' in SAP. Note: Do not provide technical IDs for the Salesforce steps, only for the SAP steps (VA01/VF01)."


  3. Data Validation & Regulatory Compliance

    Scenario - Testing a process that requires strict audit trails or specific document states.

    Input Example -  "Focus on the 'Post-Document' state. I need 5 test cases that validate the accounting impact after a 'Goods Issue'. Each test should verify that the 'Financial Document' is generated and linked to the original 'Delivery Note'."

Customized output with naming conventions

To maintain an organized Panaya repository, use the Additional User Request field to enforce a naming standard. This is the most effective way to ensure tests are easily searchable and reportable.

Examples

You can copy and paste these into the "Additional Instructions" box -

  • Standard Business Flow

    "Name all tests using the format: [Module] - [Process Name] - [Scenario Type]. Example: SAP SD - Sales Order - Positive."

  • Risk-Based Testing

    "Use a risk-focused naming convention: [Priority] | [System] | [Business Milestone].

    Example: P1 | Oracle | Ledger Reconciliation."

  • Project-Specific

    "Prefix all generated tests with the project code '2026_Upgrade'.

    Example - 2026_Upgrade: SAP MM: Goods Receipt."

Troubleshooting Your Results

If the AI gives you...

Try adjusting your prompt to...

Too many UI steps (e.g., "Open Menu")

"Focus on high-level business logic and milestones only."

Missing SAP T-Codes

"Ensure every SAP step includes its primary Transaction Code."

Too many scenarios

"Generate exactly 3 high-priority positive paths."

Vague Expected Results

"Define expected results in terms of system status or document IDs."

Combining Instructions

You can combine multiple instructions for the best results.

Example -
"Generate 5 tests. Use the naming convention [System] - [Process]. Ensure all SAP steps have T-Codes and prioritize 'Negative' scenarios related to credit limits."