Chapter 6

ATAC: Executing Software Tests

When a program that has been compiled with ATAC is executed, in addition to its normal output, it generates coverage information that is appended to a trace file. The information within this file is later used by ATAC and ATAC to produce coverage displays and reports. This chapter discusses how to manipulate the trace file and identifies problems that might occur during test execution.

6.1 Naming the Trace File

An ATAC trace file name ends with the extension .trace. If this extended name is not specified and the ATAC_TRACE environment variable is set, .trace will automatically be appended to the name set in the ATAC_TRACE environment variable. The trace file name may not begin with a hyphen.

The trace file may be written in a directory other than the current directory by setting the ATAC_DIR environment variable with a path to the desired directory. ATAC_DIR may be set to either an absolute or relative path.

6.2 Trace File Compression

6.2.1 Forcing Trace File Compression

In addition, if the nature of a program or a set of tests is such that many executions exit abnormally (e.g., via receipt of a signal), the trace file may not be compressed prior to program exit. In this case, trace file compression should periodically be forced even if ATAC_COMPRESS has not been set.

If invoked with a .trace file as it's argument, the atactm command performs trace file compression (see Section 7.8, Concerning Trace File Compression). This offers a simple way to force trace file compression, regardless of whether or not the ATAC_COMPRESS environment variable is set. For example, invoking atactm as follows:

	prompt:> atactm wordcount.trace

6.3 Temporary Trace Files

The temporary trace file will not be incorporated into the master trace file if:

• The ATAC_COMPRESS environment variable is set to 0 or no.
  
  
• The instrumented program terminates abnormally.
  
  
• The master trace file is locked. The master trace file may be locked for the following reasons:
  
  ¯ The trace file is being compressed by another program.

  ¯ The trace file is being browsed with the graphical interface.

  ¯ The lock file was not deleted due to a previous abnormal termination of the graphical interface or the atactm compression tool.

• There is not enough disk space available to incorporate the temporary trace file into the master trace file.
  
  
• The temporary trace file is not readable (e.g. the file system is not mounted) at the time the compression tool is run.
  
  

6.4 Trace File Locking

The lock is a file in the same directory as the trace file with the same name as the trace file except that the .trace suffix is replaced with .AElock.

If the lock file exists or cannot be created (due to directory permissions), tools that did not create the lock cannot edit or compress the trace file. In addition, if the graphical interface cannot create the lock and test cases are still running, the whole trace file will have to be reread every time the ``Update'' button is selected.

Normally the lock file is automatically deleted by the tool that created the lock. If the tool terminates abnormally, the lock file may not be deleted. The lock file may be deleted manually.

Note that locking is not needed to prevent trace file corruption. If it becomes necessary to ``force'' a lock, due to apparent failure of it's owner, the resulting trace may become incomplete but will not become corrupted.

6.5 Trace File Permission

In the UNIX environment, the master trace file is created with the same read/write permissions for user/group/other as the directory it is created in. Compression will preserve the read/write permissions on the trace file.

Temporary trace files are created with the same permissions as the trace file that points to it.

For example, if the directory the trace file is created in is readable and writable by user and group (0770), the trace file and temporary trace files are created readable and writable by user and group (0660) when a test is run by anyone in the group. The same trace file may be appended and/or compressed by anyone in the group.

If the ATAC_UMASK environment variable is set before linking or execution, it should be set to an octal number from 0 to 0666. The value is used to restrict the permissions on the trace file, e.g. if the directory to contain the trace file has permissions 0775 and ATAC_UMASK is 026 then the trace file is created with permission 0640.

Note that the ownership of the trace file changes to the last one who compressed it. This shouldn't matter as long as the permissions for group or other permit reading and writing.

Note also, that if a directory has the ``sticky bit'' set, files can be deleted from the directory only by the owner (regardless of file and directory permissions). This bit is often set on /tmp and /usr/tmp so that people can't delete each others temporary files even though everybody can write the directory. If this bit is set on the directory where ATAC is putting the temporary trace files, and there are multiple users writing to the same trace file, during compression, some of the temporary trace files will not get deleted. When this happens, atactm truncates the files to size zero to save disk space and as an indication that the file is no longer needed.

6.6 Parallel Test Execution

6.7 Improving Execution Speed and Saving Disk Space

The impact of ATAC on execution speed and disk space may also be reduced by limiting the scope of coverage analysis to a subset of the source files in the program under test (see Section 5.1.3, Selectively Instrumenting Software (UNIX) or Section 5.2.3, Selectively Instrumenting Software (Windows)). Using this approach, normal program performance only degrades while executing code within the source files compiled with ATAC. Coverage analysis of an entire software system is obtained by combining the results of executing identical tests on a number of source file subsets, each selectively instrumented in a complementary manner.

If testing is being conducted in a network environment where remote file systems are being accessed from a file server, execution time may also be reduced by directing the trace file to a file on a local disk (see Section 6.1, Naming the Trace File). This eliminates network file access time when writing to the trace file. If ATAC_TMP is set, it should also point to a directory on a local disk (see Section 4.1.6, ATAC_TMP).

Execution time may also be reduced by restricting the amount of detail in the coverage data. In particular, if only block coverage data is needed, execution time may be reduced by setting ATAC_BLOCKONLY to yes (see Section 4.1.1, ATAC_BLOCKONLY).

Under certain circumstances, there may be a section of code for which normal execution speed is critical, even at the cost of missing coverage data. If these sections are identified, ATAC will omit instrumentation in them. These sections will be counted as code that has not been covered. To identify code for which normal execution speed is critical, insert the word TIMECRITICAL in a comment preceding the section of code. After the section of code, insert the word NOTTIMECRITICAL in a comment. For example the play_audio function that follows must copy audio data to the output device as fast as possible to avoid gaps in the sound.

  	play_audio(file_in, audio_out)
	{  int	 n;
	   char  *buf[8000];
	   while (1) {
		/* TIMECRITICAL */
		while (buf_level(audio_out) < HIGH_WATER){
		       n = read(file_in, buf,8000);
		       if (n == 0)
		           return;
		       write(audio_out, buf, n);
		       }
		       /* NOTTIMECRITICAL */
	  	       usleep(100000);
		}
	}

6.8 Explaining Run-Time Errors

• The program compiled with ATAC will create or append to a trace file (see Section 6.1, Naming the Trace File).
  
  
• The program compiled with ATAC will execute somewhat more slowly than the original program. If the program is timing sensitive, this may cause other behavioral differences (see Section 3.5, What Will Using ATAC Cost You? and Section 6.7, Improving Execution Speed and Saving Disk Space).
  
  
• Programs that appear to behave correctly sometimes contain memory violations which, by chance, do not interfere with program execution (e.g., accessing unallocated memory locations). When ATAC is used, these violations may begin to interfere with program execution and result in program failure. Conversely, memory violations that would normally result in program failure may be masked when using ATAC.