[iOS] 0% code coverage on new code

Report a bug
,
#1

Summary

We are using SonarCloud with my new employer and I’m facing a pretty strange issue with code coverage calculation on new code.

My Objective-C project is up and running, I’ve manually analyzed the master and develop branches with sonar-scanner. The master branch has 0% coverage from Xcode and the develop branches reaches 98.8%. SonarCloud, for both branches, reported that the coverage was 0% since I didn’t run the slather command to give them a generic coverage file.

So I’ve recently opened a PR from develop to fix some bugs & code smells reported by SonarCloud and I’ve also added a new UT to reach the 100% coverage on Xcode. And since the CI is now handling the whole thing, it runs the slather action by itself with fastlane.

Unfortunately, I can’t merge my PR because SonarCloud reports a 0% coverage on new code:

Screenshot 2021-04-15 at 14.01.03
Screenshot 2021-04-15 at 14.01.032540×1118 199 KB

At some point, adding a dummy method fixed the issue but was compliant with our production workflow and it doesn’t work anymore :sob:

Here’s a screenshot of the sonarqube-generic-coverage.xml file showing that there is coverage:

Screenshot 2021-04-15 at 14.42.01
Screenshot 2021-04-15 at 14.42.01607×951 40.7 KB

And here’s the content of the sonar-project.properties:

sonar.cfamily.build-wrapper-output=bw-output
sonar.sourceEncoding=UTF-8
sonar.coverageReportPaths=output/sonarqube-generic-coverage.xml

Maybe I am doing something wrong, do you have any insight or ideas about this?

Thanks in advance.

Damien

Miscellaneous

Version: SonarCloud (Developer 500k)

#2

Hey there.

If all was fine with your manual branch analyses but you’re facing an issue when it’s executed in the CI… I’m guessing your manual runs were better configured!

In your analysis (scanner) logs you should find some log lines like this:

INFO: Sensor Generic Coverage Report
INFO: Parsing /Users/colin/Source/sonar-scanning-examples/swift-coverage/swift-coverage-example/sonarqube-generic-coverage.xml
INFO: Imported coverage data for 0 files
INFO: Coverage data ignored for 3 unknown files, including:
/Users/colin/Source/sonar-scanning-examples/swift-coverage/swift-coverage-example/swift-coverage-exampleTests/swift_coverage_exampleTests.swift
/Users/colin/Source/sonar-scanning-examples/swift-coverage/swift-coverage-example/swift-coverage-example/AppDelegate2.swift
/Users/colin/Source/sonar-scanning-examples/swift-coverage/swift-coverage-example/swift-coverage-example/AppDelegate.swift
INFO: Sensor Generic Coverage Report (done) | time=65ms

What’s it telling you? :smiley:

#3

Hi @Colin_SonarSource,

Thanks for taking the time to help me :slightly_smiling_face:

Here are the log I’m getting from the sonar-scanner ran by the CI:

INFO: Sensor Generic Coverage Report
INFO: Parsing /Users/travis/build/XXX/XXX/output/sonarqube-generic-coverage.xml
INFO: Imported coverage data for 12 files
INFO: Coverage data ignored for 5 unknown files, including:
travis/test_derived_data/Build/Products/Debug-iphonesimulator/XCFrameworkIntermediates/XXX/XXX.framework/Headers/XXX.h
travis/test_derived_data/Build/Products/Debug-iphonesimulator/OCMock/OCMock.framework/Headers/OCMArg.h
travis/test_derived_data/Build/Products/Debug-iphonesimulator/OCMock/OCMock.framework/Headers/OCMStubRecorder.h
travis/test_derived_data/Build/Products/Debug-iphonesimulator/OCMock/OCMock.framework/Headers/OCMockMacros.h
travis/test_derived_data/Build/Products/Debug-iphonesimulator/XCFrameworkIntermediates/XXX/XXX.framework/Headers/XXX.h

So it parses the relevant files (the .m files) but discovers no coverage since there’s no new code so to say.

My best guess is that since there’s actually no new code, it means that there’s no new code coverage but it should not block the pull request.

#4

Hm. :confused: If it was actually finding any coverage at all, I would expect to see some coverage listed for Estimated after merge in your screenshot.

That’s an interesting point. So the 1.3k New Lines to Cover seeen in your screenshot is just… wrong? What code changes do you expect to see in the PR, and how does it compare to the Code tab of your pull request?

#5

Here are the stats for the develop branch:

Screenshot 2021-04-15 at 15.12.10
Screenshot 2021-04-15 at 15.12.102462×152 8.41 KB

And here, the stats for the new branch:

Screenshot 2021-04-15 at 15.12.26
Screenshot 2021-04-15 at 15.12.262456×162 10.1 KB

As you can see, the “new code” is only tests, I have even removed some useless code from the develop branch.

#6

Okay! In that case, I would make sure that you have set sonar.tests correctly in your sonar-project.properties file so those Tests aren’t counted as New Lines of Code (and as code to be covered).

It doesn’t quite explain 0% estimated coverage after merge. But what I would suggest is testing that out on a new branch where you’ve actually added some source code.

Is your CI responsible for running your main branch analysis yet, or only PRs? You mentioned before you had coverage successfully imported on some manual runs.

#7

Yup, it is configured as follow:

sonar.sources=Framework/Sources
sonar.tests=Framework/Tests

The project file system is as follow:

  • Framework
    • Sources
      • .h & .m files
    • Tests
      • .m files

What is really weird is that by adding dummy code, I managed to make it work and it went from 0% to 100% coverage on new code last time I tried… But it seems that I cannot make it work again.

I’ll do more testing with this in the meantime.

#9

Yup, it runs on all branches master, develop and all commits made for PRs.

I’m currently waiting on our CI to complete a run with a new set of dummy method with their own set of tests (yay false coverage :laughing:).

#10

Welp… nothing new, coverage is still at a whopping 0% even though Xcode reports a 100% and the generic coverage file is stating that each single line is covered :weary: