Cobol rule cobol:COBOL.UnusedDataItem should not take into account EXTERNAL variables

SonarQube Server / Community Build
1

Must-share information (formatted with Markdown):

  • Enterprise Edition Version 9.9.1 (build 69595)
  • how is SonarQube deployed: no idea (on https://web.sonarqube.axa.be/)
  • exclude EXTERNAL variables from that rule
  • reading docs to find out anything about similar issue

current rule is

DATA DIVISION.
WORKING-STORAGE SECTION.
  01 PERSON.        *> Compliant as sub data item FIRST_NAME is used
    02 FIRST_NAME PIC X(21).
    02 LAST_NAME PIC X(21).

  01 ADDR.   *> Noncompliant as no data item in this block is used
    02 STREET PIC X(50).
    02 TOWN PIC X(50).

Would be nice to add exclusion for External variables (or add an option to allow or Not this exclusion)

  01 CONTACTPOINT EXTERNAL.        *> Compliant as EXTERNAL definition is used
    02 SPOC_NAME PIC X(21).
    02 SPOC_ADDRESS PIC X(21).

In our company, we defined lot of our EXTERNAL in our MAIN programs, to ensure they won’t get lost (marked as Freed from External dedicated memory) at the time we exit the program that first declared an external resource. They are not dead code, as removing them will cause data loss. This is due to IMS memory management, in opposition to CICS.

2

Hi,

Thanks for your feedback.

We can consider adding an exception to the rule.
However, I’d like to understand better the problem.
I never heard of the EXTERNAL clause before.
How is such a data item used in your program?
Could you please share more code to explain why removing this data item would break something?

Thanks a lot.

3

Hello Pierre-Yves,

First, Thanks for your answer.

EXTERNAL Clause is used to define group element in a specific portion of memory – the External Table – which is shared amongst the running Job.

The first time a variable is declared with EXTERNAL clause, system first looks in this External table if same variable name has already been defined and, if this is the case, it will check the length of the declared variable, as it MUST match what have been reserved in the External Table. If not already reserved, it will add its name, length and address in the External Table and retains the program ID that first declared the External.

Once the program that declared the first some External exits, the system marks all the External that have been declared in that program as free. (logical delete, contains stay in place as long as not reused by another external)

The big advantage of external is not to have to specify all parameters via Linkage/procedure using amongst lot of programs and sub-programs.

Advantage is obvious : Program 1 is calling program 2 which is calling program 3 … till program 9… Program 9 need extra data, you don’t need to add these data and add them to each Procedure Using/ Linkage and CALL using …

Rules are to be applied when using External : Variable Name must be unique across our organization. Use only external for Business shared data used by several routines, never use External for program input/parameter or options…

In our main “business” programs, we have a lot of data with External that are all declared in top level program.

They are referenced and/or modified by sub programs.

All External content are kept as long as we don’t exit the top level program that declares these externals.

You can read more technical details on that topic here

So any data definition with External clause couldn’t be validated as unused at compile time – this can be done only at run time.

Tell me if you want more info.

Cheers

Stef

5

Hi Stef,

Thanks a lot for your message and sorry for replying somewhat late.

Your explanations definitely help.
What I still don’t understand is why a program would declare a variable as external if it doesn’t use it either to read it or write it.

In your example with the chain from program 1… program 9, I understand that a variable could be:

  • declared as external and written in program 1
  • declared as external and read in program 9
  • unused in program 2 and others

Then, I believe that it would make sense to detect the variable as unused if it was declared in program 2.
Am I missing something?

Thanks again,
Pierre-Yves

7

Hi Pierre-Yves,

Sorry for late replay, was unavailable till 1st of March.

To get a better view on this topic, please have a look to following picture :

image001.png1226×724 126 KB

When a variable is declared with EXTERNAL clause, system first looks in this External table if same variable name has already been defined.

Here, as it’s the first time this external name is encountered at Run Time, it will add its name, length and address in the External Table and retains the program ID that first declared the External. (PROGRAM1)

Once the program (PROGRAM1) that first declared these External exits, the system marks all these External (E1 E2 E3) that have been declared in that program as free. (Logical delete, contains stay in place as long as not reused by another external)

PROGRAM1 is declaring all externals that will be used in current application during one LUW. (job)

It doesn’t use them, just reserving all of them. The first time a variable is declared with EXTERNAL clause, system first looks in this External table if same variable name has already been defined : when new one, it will add its name, length and address in the External Table and retains the program ID that first declared this External.

PROGRAM2 is also declaring E1 external, System will check the External table and found it as it was previously reserved by PROGRAM1, System checks so it matches the length of the declared variable in its table, and as it matches, it will reuse same address & by so, same content.

When leaving the PROGRAM2, E1 external is not marked as FREE as the program hat initially declared E1 external is PROGRAM1.

PROGRAM6 is declaring E1 external, System will check the External table and found it as it was previously reserved by PROGRAM1, System checks so it matches the length of the declared variable in its table, and as it matches, it will reuse same address & by so, content that also have been modified by PROGRAM2.

Unused/unreferenced external clean-up can be done at run time only :

  • If we remove unreferenced declarations of E2 & E3 from PROGRAM4, you’re right : it won’t change anything in this case – all external have been defined in PROGRAM1
  • If we remove all unreferenced declarations of E1 E2 E3 external from PROGRAM1, it WILL cause DATA issue : Data consistency won’t be ensured between PROGRAM2 and PROGRAM6 as E1 external will be marked as free (logical delete) when ending the PROGRAM2 (first program to declare it at RUN TIME). So, you have no idea of what will be the content of external E1 when executing PROGRAM6. Content could be kept or it could be reused by IMS system when memory space is needed – so we can use wrong data (worse case) or we can get an abend (best case).

As SonarQube analysis is done program per program, and can’t be done at RUN TIME, you can just detect an EXTERNAL field is not referenced in that program, but it’s not sufficient to recommend a deletion, as this recommendation could only be valid for intermediate programs.

Hope this is a bit clearer, don’t hesitate to ask for more in case

Cheers

Stef

8

Hi Stef,

Sorry for the late reply.
And thanks a lot for the detailed explanation!
It definitely helps to understand your use case.
I created SONARCOBOL-1701 to track this issue.

Thanks a lot for your feedback!
Pierre-Yves