[go: up one dir, main page]
Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[🐛 Bug]: wdio/allure-reporter - Summary.json file showing the total test steps account instead of test case count #13623

Closed
3 tasks done
varshanharshank opened this issue Sep 20, 2024 · 3 comments
Closed
3 tasks done

[🐛 Bug]: wdio/allure-reporter - Summary.json file showing the total test steps account instead of test case count #13623

varshanharshank opened this issue Sep 20, 2024 · 3 comments
Labels
Bug 🐛 Reproducible Example Missing

Comments

@varshanharshank
Copy link

Have you read the Contributing Guidelines on issues?

WebdriverIO Version

8.35.1

Node.js Version

v22.5.1

Mode

Standalone Mode

Which capabilities are you using?

capabilities: [
    {
      // capabilities for local Appium web tests on an Android Emulator
      'appium:platformName': 'ios',
      'appium:deviceName': 'iPhone 12',
      'appium:platformVersion': '16.3.1',
      'appium:automationName': 'XCUITest',
      'appium:udid': '00008101-000221',
      'appium:app': path.join(
        process.cwd(),
        'app/ios/' + 'test.ipa'
      ),
      'appium:appPackage': ApplicationConstants.MOBILE_APP.IOS.APP_PACKAGE,
      'appium:xcodeOrgId': 'TTSS6W',
      'appium:xcodeSigningId': 'Apple Development',
      'appium:autoAcceptAlerts': true,
      'appium:noReset': true,
    }]

What happened?

My project was setup with Appium + WebdriverIO + cucumber + Allure report.

  1. Summary.json (allure-report/widgets/summary.json)
    I can successfully run the cucumber scenarios and generate the allure report In the summary.json file (allure-report/widgets/summary.json) showing the details as below.

Here total is showing the cucumber test steps count instead of the test case count. In this case, we are unable to fetch exact number of test cases are passed and failed.

{
  "reportName": "Allure Report",
  "testRuns": [],
  "statistic": {
    "failed": 0,
    "broken": 0,
    "skipped": 0,
    "passed": 8,
    "unknown": 0,
    "total": 8
  },
  "time": {
    "start": 1726783025063,
    "stop": 1726783298539,
    "duration": 273476,
    "minDuration": 3,
    "maxDuration": 90938,
    "sumDuration": 128076
  }
}
  1. suites.json (allure-report/data/suites.json)
    This file name attribute is displayed with a combination of Featue name with Scenario Outline. It should be displayed separately. In the last index, the name attribute always is showing empty(not sure why this is showing up).

Feature: Login feature in MyTest app in iOS
@smoke @SSOLoginStatus
Scenario Outline: MyTest Login using , in iOS Device
Given Load the PlusHome Application for iOS
Then Verify Sign in page is displayed for iOS
And Enter the "" and "" for iOS
And Tab the Sign In Button for iOS
Then Enter the "" and Complete Secret Question page if it is displayed for iOS
Then Click Accept Term button in the Agreement Page for iOS if it is displayed

   Examples:
        | userId                 | password    | secrectAnswer | accountType |
        | home@gmail.com              | Home   | Test       | CH1     |

After running the above scenario, we observed the data/summary.json file, it shows below information.
data/summary.json.Here we noticed that

  1. name attribute is displayed with a combination of Featue name with Scenario Outline. It should be displayed separately.
  2. In the last index, the name attribute always is showing empty(not sure why this is showing up).
{
 "uid": "98d3104e051c652961429bf95fa0b5d6",
 "name": "suites",
 "children": [
   {
     "name": "Login feature in MyTest app in iOS: PlusHome Login using home@gmail.com,CH1 in iOS Device",
     "children": [
       {
         "name": "Given Load the MyTest Application for iOS",
         "uid": "4c5c554aada4e55a",
         "parentUid": "ea3540a7ea4bc7d89c7f879956cd7646",
         "status": "passed",
         "time": {
           "start": 1726783025063,
           "stop": 1726783026131,
           "duration": 1068
         },
         "flaky": false,
         "newFailed": false,
         "newPassed": false,
         "newBroken": false,
         "retriesCount": 0,
         "retriesStatusChange": false,
         "parameters": ["iPhone 12"],
         "tags": []
       },
       {
         "name": "Then Verify Sign in page is displayed for iOS",
         "uid": "f7a1e6c44ae907db",
         "parentUid": "ea3540a7ea4bc7d89c7f879956cd7646",
         "status": "passed",
         "time": {
           "start": 1726783026131,
           "stop": 1726783038333,
           "duration": 12202
         },
         "flaky": false,
         "newFailed": false,
         "newPassed": false,
         "newBroken": false,
         "retriesCount": 0,
         "retriesStatusChange": false,
         "parameters": ["iPhone 12"],
         "tags": []
       },
       {
         "name": "And Enter the \"home@gmail.com\" and \"Home&\" for iOS",
         "uid": "955b3dd0b2fbfb64",
         "parentUid": "ea3540a7ea4bc7d89c7f879956cd7646",
         "status": "passed",
         "time": {
           "start": 1726783038333,
           "stop": 1726783042842,
           "duration": 4509
         },
         "flaky": false,
         "newFailed": false,
         "newPassed": false,
         "newBroken": false,
         "retriesCount": 0,
         "retriesStatusChange": false,
         "parameters": ["iPhone 12"],
         "tags": []
       },
       {
         "name": "And Tab the Sign In Button for iOS",
         "uid": "41d65b88a8d648fe",
         "parentUid": "ea3540a7ea4bc7d89c7f879956cd7646",
         "status": "passed",
         "time": {
           "start": 1726783042842,
           "stop": 1726783046415,
           "duration": 3573
         },
         "flaky": false,
         "newFailed": false,
         "newPassed": false,
         "newBroken": false,
         "retriesCount": 0,
         "retriesStatusChange": false,
         "parameters": ["iPhone 12"],
         "tags": []
       },
       {
         "name": "Then Enter the \"Test\" and Complete Secret Question page if it is displayed for iOS",
         "uid": "6695fdbe45099003",
         "parentUid": "ea3540a7ea4bc7d89c7f879956cd7646",
         "status": "passed",
         "time": {
           "start": 1726783046415,
           "stop": 1726783060648,
           "duration": 14233
         },
         "flaky": false,
         "newFailed": false,
         "newPassed": false,
         "newBroken": false,
         "retriesCount": 0,
         "retriesStatusChange": false,
         "parameters": ["iPhone 12"],
         "tags": []
       },
       {
         "name": "Then Click Accept Term button in the Agreement Page for iOS if it is displayed",
         "uid": "fb49372d23a9cb3e",
         "parentUid": "ea3540a7ea4bc7d89c7f879956cd7646",
         "status": "passed",
         "time": {
           "start": 1726783060650,
           "stop": 1726783151588,
           "duration": 90938
         },
         "flaky": false,
         "newFailed": false,
         "newPassed": false,
         "newBroken": false,
         "retriesCount": 0,
         "retriesStatusChange": false,
         "parameters": ["iPhone 12"],
         "tags": []
       },
       {
         "name": "Then Verify User landed to PlusHome home page for iOS and track Status based on \"home@gmail.com\" and \"CH1\"",
         "uid": "6cbc1f3f70f4a425",
         "parentUid": "ea3540a7ea4bc7d89c7f879956cd7646",
         "status": "passed",
         "time": {
           "start": 1726783151589,
           "stop": 1726783153139,
           "duration": 1550
         },
         "flaky": false,
         "newFailed": false,
         "newPassed": false,
         "newBroken": false,
         "retriesCount": 0,
         "retriesStatusChange": false,
         "parameters": ["iPhone 12"],
         "tags": []
       },
       {
         "name": "",
         "uid": "3af0bc058afc5f57",
         "parentUid": "ea3540a7ea4bc7d89c7f879956cd7646",
         "status": "passed",
         "time": {
           "start": 1726783298536,
           "stop": 1726783298539,
           "duration": 3
         },
         "flaky": false,
         "newFailed": false,
         "newPassed": false,
         "newBroken": false,
         "retriesCount": 3,
         "retriesStatusChange": false,
         "parameters": ["iPhone 12"],
         "tags": []
       }
     ],
     "uid": "ea3540a7ea4bc7d89c7f879956cd7646"
   }
 ]
}

//package.json

{
"name": "My testApp",
"version": "1.0.0",
"description": "testApp",
"devDependencies": {
"@types/fs-extra": "^11.0.4",
"@types/unzipper": "^0.10.9",
"@types/xml2js": "^0.4.14",
"@typescript-eslint/eslint-plugin": "^6.2.0",
"@typescript-eslint/parser": "^6.2.0",
"@wdio/allure-reporter": "^8.34.0",
"@wdio/appium-service": "^8.35.1",
"@wdio/cli": "^8.35.1",
"@wdio/cucumber-framework": "^8.35.0",
"@wdio/local-runner": "^8.35.1",
"@wdio/spec-reporter": "^8.32.4",
"allure-commandline": "^2.27.0",
"eslint": "^8.57.0",
"globals": "^15.0.0",
"prettier": "^3.2.5",
"ts-node": "^10.9.1",
"typescript": "^5.1.6"
},

"dependencies": {
"appium-uiautomator2-driver": "^3.0.1",
"axios": "^1.6.8",
"fs-extra": "^11.2.0",
"https": "^1.0.0",
"path": "^0.12.7"
}
}

What is your expected behavior?

  1. Summary.json (allure-report/widgets/summary.json)
    It should provide how many test scenarios are passed ,failed,skipped... instead of counting cucumber test step count

  2. suites.json (allure-report/data/suites.json)
    a. It should show feature name and test case name should be separately
    b. In the childer array (last index) name attribute is showing empty. This should not be present.

How to reproduce the bug.

I am unable share the project due to this is private repo.

i ran the below command to execute the cucumber scenario

npx wdio run ./wdio.conf.ts --cucumberOpts.tags='@SSOLoginStatus'

Run the below command for generate the report

npx allure generate reports/allure-results --clean -o reports/allure-report && npx allure open reports/allure-report

Relevant log output

N/A

Code of Conduct

  • I agree to follow this project's Code of Conduct

Is there an existing issue for this?

  • I have searched the existing issues
@varshanharshank varshanharshank added Bug 🐛 Needs Triaging ⏳ No one has looked into the issue yet labels Sep 20, 2024
@BorisOsipov
Copy link
Member

Hi @varshanharshank what is your wdio allure reporter config? do you use useCucumberStepReporter:true option?

To repro this or at least investigate we need repro. please don't share text of features\configs and etc. Just create a simple repo on github with all the code and share the link.

Also, we don't need suites.json\summary.json - we can generate it ourselves - share content of allure-results folder instead.

@BorisOsipov BorisOsipov added Reproducible Example Missing and removed Needs Triaging ⏳ No one has looked into the issue yet labels Sep 20, 2024
@wdio-bot
Copy link
Contributor

Thanks for raising this issue 🙏

Unfortunately we can't help you without a reproducible example in this matter. Please read our contributing guidelines on how to create a reproducible example. If you can't provide a reproducible example we will close this issue in 7 days.

If you have any questions please reach out to us on our Discord channel. We are happy to help you out there.

@christian-bromann
Copy link
Member

Closing due to inactivity.

@christian-bromann christian-bromann closed this as not planned Won't fix, can't repro, duplicate, stale Oct 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Bug 🐛 Reproducible Example Missing
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
4 participants
@christian-bromann @BorisOsipov @wdio-bot @varshanharshank