Loss data possibility

Hello,

Refering to Fixed Issues 0 of 1 shown

I have investigated a project, and find out that some issues has been loss from my project

Here is the step:
I analyze my project on 24 January 2019, and it runs normally. There are issues, and fixed issues.
I analyze my project on 31 January 2019, and it also runs normally. But my unresolved issues is different and so is my fixed issues. So I check the data and found out there are some loss data.
From SonarQube Web it will shows 0 of 13 issues but there is no issues on the list. I know that there should be 13 fixed issues because on 24 January 2019 the issues has been fixed. It should not be delete yet because I set my Database Cleaner for closed issues for 90 days.

I checked my DB and can not find the issues anymore.

I have restarted after delete $SONARQUBE_HOME/data/es and there are no changes in these issues.

Can you help me? If you need me to provide more information, please feel free to reply these topic. Thanks.

Hi,

By “Fixed” issues, do you mean status Closed and resolution Fixed, or status Resolved and resolution Fixed?

Also, do your overall issue counts change? And finally, is your overall lines of code in the project approximately the same from one analysis to the next? If it dropped significantly, it would indicate that your issue counts changed because code that was included in the first analysis was omitted from the second.

 
Ann

By “Fixed” issues, do you mean status Closed and resolution Fixed, or status Resolved and resolution Fixed?
*Fixed issues here is status Closed and resolution Fixed.

Also, do your overall issue counts change?

  • the counts changes, there are around 1000 thousand issues difference

And finally, is your overall lines of code in the project approximately the same from one analysis to the next?

  • Overall line of codes doesn’t change.

If it dropped significantly, it would indicate that your issue counts changed because code that was included in the first analysis was omitted from the second.

  • If the issues changed because the codes, it should closed or create a new issues. But this one is deleted, because I can not find it in DB, but it still counted when I try to search it using API (0 of 13 issues shown). Anyway, the issues should not be deleted but closed as fixed. Because my DB cleaner will not deleted for 90 days, and the closed issues is still in within 90 days.

Hi,

Is this happening to all projects or just 1? Has the quality profile changed lately?

 
Ann

There a no changes in Quality Profiles.
So far, I have found two projects with these kind of issues. I have not search all of my projects thoroughly.

:slightly_frowning_face:

Hi,

This is the first report I’ve ever seen of anything like this. Do you have anything interesting or unusual in the analysis logs of those projects for the analysis where issues disappeared? Also, can you double-check the housekeeping settings at the project level for those two projects?

 
Ann

  • I have double checked it and it still set as 90 in both of the projects at the project level.
  • I am still trying to look for the log. Right now, I have some technical problem, so I can not get the log yet. As far as I know, I am using jenkins and there are no changes in jenkins codes, so I still assumed there should be no unusual activity in analyzing the project.

I will post more information if I have get the log.

@ganncamp

Hello, Sorry that it took some time to get the log. Is there a way that the log can only be viewed by your team since it contains sensitive information. Thanks.

Hi,

Feel free to redact the log before you post it, or I guess you can send it to me privately: ann dot campbell at sonarsource.com.

 
Ann

Hi Fanny,

I’ve received your log, and noticed a few things:

  • it doesn’t begin at the beginning. Ideally, you’ll re-send a file that starts with the analysis command
  • you appear to be using sonar.language=java. First, that long-deprecated parameter will be dropped soon (likely in 7.7). Second, is that parameter used every time? If its application is inconsistent, that could easily explain intermittent issues.
  • you’re also setting exclusions. Are they set via the UI (and so reliably applied every time) or set on the command line or in the analysis properties? Again, if they’re not applied consistently, this could explain issues that come and go
  • there are warnings about missing classes. SonarJava requires access to class files to provide a full analysis. If classes are available only intermittently, this could also explain occasionally-missing issues

 
Ann

@ganncamp

  • Okay, I will resend it again, but I will still redact the sensitive information.

  • I always used this parameter. Actually, I am using Jenkins to run job, so there are no changes in how I scan my project.

  • I set it on analysis properties, and they are applied consistently. I only excluded model/java.

  • The missing classes are third party’s. If SonarJava can not access a class, I thought it will closed the issues, not delete the issues, isn’t it?

Hi,

I still don’t see the actual build/analysis command in the new log you sent. Nonetheless, nothing stands out to me. I have one last idea which is to double-check your project history. Issues would be deleted after they’ve aged out of the system or when the project was deleted. So does your project history go back as far as you expect, or is it possible the project has been deleted and re-created?

 
Ann

Hi @ganncamp

I am using mvn sonar:sonar as analysis command. And it has been used for this project without any changes.

I have double checked my project history and number days of the issues will be cleaned. I even recreate this issues by cloning the project, then analyze the project as I analyze the original. And it still happens.

  • I don’t think the problem is the issues age, it still within the number of days before it is cleaned. (Even if it is already aged out, it should not display 0 of 1 issues.)
  • This project has never been deleted.

Is there any other suggestions that I could try? Or any idea why this issue is happening?

:cry:

Hi,

If you can re-create the problem then…

  • can you provide a small reproducer project and instructions for reproducing? (I’m anticipating a “no” here, but I have to ask.)
  • can you identify any pattern in the issues that go missing?

 
Ann

@ganncamp

I am sorry, I can’t provide the data. :cry:

I can’t identify the pattern. I am still trying to find out, I’ll let you know if I find the pattern…

Thank you