![]() PVS-Studio Static Code Analyzer for 64-bit and parallel C/C++ code
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
![]() ![]() ![]() ![]() ![]()
11.03.2010
Parallel notes N4 - continuing to study OpenMP constructs In this post we will continue to introduce you into OpenMP technology and tell you about some functions and new directives.»
02.03.2010
Parallel notes N3 - base OpenMP constructs Now we would like to start introducing you into OpenMP technology and show you the ways of using it.»
28.02.2010
In what way can C++0x standard help you eliminate 64-bit errors Programmers see in C++0x standard an opportunity to use lambda-functions and other entities I do not quite understand :).» ![]()
10.12.2009
PVS-Studio FAQ This paper contains some questions and answers about PVS-Studio code analyzer by OOO "Program Verification Systems".»
09.12.2009
VivaCore FAQ This paper contains some questions and answers about VivaCore C/C++ code analysis library by OOO "Program Verification Systems"»
23.11.2009
PVS-Studio: using the function "Mark as False Alarm"
The article describes and demonstrates by an example the use of PVS-Studio 3.40 new function "Mark as False Alarm". » ![]() |
64-bit Development![]() Seven Steps of Migrating a Program to a 64-bit SystemAndrey Karpov
| ||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
| Type | Type's size on x32 / x64 platform | Note |
|---|---|---|
| int | 32 / 32 | Basic type. On 64-bit systems remains 32-bit. |
| long | 32 / 32 | Basic type. On 64-bit Windows systems remains 32-bit. Keep in mind that in 64-bit Linux systems this type was extended to 64-bit. Don't forget about it if you develop code which should be compiled for Windows and Linux systems. |
| size_t | 32 / 64 | Basic unsigned type. The type's size is chosen in such a way that you could write the maximum size of a theoretically possible array into it. You can safely put a pointer into size_t type (except for pointers to class functions, but this is a special case). |
| ptrdiff_t | 32 / 64 | Similar to size_t type but this is a signed type. The result of the expression where one pointer is subtracted from the other (ptr1-ptr2) will have ptrdiff_t type. |
| Pointer | 32 / 64 | The size of the pointer directly depends on the platform's size. Be careful while converting pointers to other types. |
| __int64 | 64 / 64 | Signed 64-bit type. |
| DWORD | 32 / 32 | 32-bit unsigned type. In WinDef.h is defined as:typedef unsigned long DWORD; |
| DWORDLONG | 64 / 64 | 64-bit unsigned type. In WinNT.h is defined as:typedef ULONGLONG DWORDLONG; |
| DWORD_PTR | 32 / 64 | Unsigned type in which a pointer can be placed. In BaseTsd.h is defined as:typedef ULONG_PTR DWORD_PTR; |
| DWORD32 | 32 / 32 | 32-bit unsigned type. In BaseTsd.h is defined as:typedef unsigned int DWORD32; |
| DWORD64 | 64 / 64 | 64-bit unsigned type. In BaseTsd.h is defined as:typedef unsigned __int64 DWORD64; |
| HALF_PTR | 16 / 32 | A half of a pointer. In Basetsd.h is defined as:#ifdef _WIN64 typedef int HALF_PTR;#else typedef short HALF_PTR;#endif |
| INT_PTR | 32 / 64 | Signed type in which a pointer can be placed. In BaseTsd.h is defined as:#if defined(_WIN64) typedef __int64 INT_PTR; #else typedef int INT_PTR;#endif |
| LONG | 32 / 32 | Signed type which remained 32-bit. That's why in many cases LONG_PTR now should be used. In WinNT.h is defined as:typedef long LONG; |
| LONG_PTR | 32 / 64 | Signed type in which a pointer can be placed. In BaseTsd.h is defined as:#if defined(_WIN64) typedef __int64 LONG_PTR; #else typedef long LONG_PTR;#endif |
| LPARAM | 32 / 64 | Parameter for sending messages. In WinNT.h is defined as:typedef LONG_PTR LPARAM; |
| SIZE_T | 32 / 64 | Analog of size_t type. In BaseTsd.h is defined as:typedef ULONG_PTR SIZE_T; |
| SSIZE_T | 32 / 64 | Analog of ptrdiff_t type. In BaseTsd.h is defined as:typedef LONG_PTR SSIZE_T; |
| ULONG_PTR | 32 / 64 | Unsigned type in which a pointer can be placed. In BaseTsd.h is defined as:#if defined(_WIN64) typedef unsigned __int64 ULONG_PTR;#else typedef unsigned long ULONG_PTR;#endif |
| WORD | 16 / 16 | Unsigned 16-bit type. In WinDef.h is defined as:typedef unsigned short WORD; |
| WPARAM | 32 / 64 | Parameter for sending messages. In WinDef.h is defined as:typedef UINT_PTR WPARAM; |
Table 3. Types to be noted while porting 32-bit programs on 64-bit Windows systems.
If you think that after correcting all the compilation errors you will get a long-expected 64-bit application we have to disappoint you. The most difficult is just ahead. At the stage of compilation you will correct the most explicit errors which the compiler had managed to detect and which mostly relate to impossibility of implicit type conversion. But this is only a small part of the problem. Most errors are hidden. From the viewpoint of the abstract C++ language these errors look safe and are disguised by explicit type conversions. The number of such errors is much larger than the number of errors detected at the stage of compilation.
You shouldn't set your hopes on /Wp64 key. This key is often presented as a wonderful means of searching 64-bit errors. In reality /Wp64 key just allows you to get some warning messages concerning incorrectness of some code sections in 64-bit mode while compiling 32-bit code. While compiling 64-bit code these warnings will be shown anyway. And that's why /Wp64 key is ignored when compiling a 64-bit application. And surely this key won't help in search of hidden errors [11].
Let's consider several examples of hidden errors.
The simplest but none the easiest for detection error class relates to explicit type conversions when significant bits are cut. A popular example is conversion of pointers to 32-bit types when transferring them into functions such as SendMessage:
MyObj* pObj = ... ::SendMessage(hwnd, msg, (WORD)x, (DWORD)pObj); |
Here the explicit type conversion is used to turn a pointer into a numeric type. For a 32-bit architecture this example is correct as the last parameter of SendMessage function has LPARAM type which coincides with DWORD on a 32-bit architecture. For a 64-bit architecture DWORD is incorrect and must be replaced with LPARAM. LPARAM type has sizes of 32 or 64 bits depending on the architecture.
This is a simple case but type conversion often looks more complicated and it is impossible to detect it using the compiler's warnings or search through the program text. Explicit type conversions suppress the compiler's diagnosis as they are intended for this very purpose - to tell the compiler that the type conversion is correct and the programmer is responsible for the code's safety. Explicit search won't help as well. Types can have non-standard names (defined by the programmer through typedef), and the number of methods to perform explicit type conversion is also large. To safely diagnose such errors you must use only a special toolkit such as Viva64 or PC-Lint analyzers.
The next example relates to implicit type conversion when significant bits are also lost. fread function's code performs reading from the file but it is incorrect when trying to read more than 2 GB on a 64-bit system.
size_t __fread(void * __restrict buf, size_t size, size_t count, FILE * __restrict fp); size_t fread(void * __restrict buf, size_t size, size_t count, FILE * __restrict fp) { int ret; FLOCKFILE(fp); ret = __fread(buf, size, count, fp); FUNLOCKFILE(fp); return (ret); } |
__fread function returns size_t type but int type is used to store the number of the bytes read. As a result at large sizes of read data the function can return a false number of bytes.
You can say that it is an illiterate code for beginners, that the compiler will announce about this type conversion and that this code is actually easy to find and to correct. This is in theory. And in practice everything may be quite different in cases of large projects. This example is taken from FreeBSD source code. The error was corrected only in December 2008! Note that the first (experimental) 64-bit version of FreeBSD was released in June 2003.
This is the source code before it has been corrected:
http://www.freebsd.org/cgi/cvsweb.cgi/src/lib/libc/stdio/fread.c?rev=1.14
And this is the corrected variant (December 2008):
http://www.freebsd.org/cgi/cvsweb.cgi/src/lib/libc/stdio/fread.c?rev=1.15
It is easy to make an error in the code while working with separate bits. The following error type relates to shift operations. Here is an example:
ptrdiff_t SetBitN(ptrdiff_t value, unsigned bitNum) { ptrdiff_t mask = 1 << bitNum; return value | mask; } |
This code works well on a 32-bit architecture and allows you to set bits with numbers 0 to 31 to unity. After porting the program on a 64-bit platform you will need to set bits 0 to 63. But this code will never set bits 32-63. Pay attention that "1" has int type and when a shift at 32 positions occurs an overflow will take place as shown in Figure 5. Whether we will get 0 (Figure 5-B) or 1 (Figure 5-C), as a result, depends on the compiler's implementation.

To correct the code we need to make "1" constant of the same type as mask variable:
ptrdiff_t mask = ptrdiff_t(1) << bitNum; |
Also pay attention that the incorrect code leads to one more error. When setting 31 bits on a 64-bit system the result of the function will be the value 0xffffffff80000000 (see Figure 6). The result of 1 << 31 expression is the negative number -2147483648. In a 64-bit integer variable this number is presented as 0xffffffff80000000.

Magic constants, i.e. numbers with the help of which the size of this or that type is defined, can cause a lot of troubles. A proper decision is to use sizeof() operators for these purposes, but in a large program an old code section can still be hidden where, as programmers believe, the pointer's size is 4 bytes and in size_t it is always 32 bits. Usually such errors look as follows:
size_t ArraySize = N * 4; size_t *Array = (size_t *)malloc(ArraySize); |
Figure 4 shows the basic numbers with which you should work with caution while migrating on a 64-bit platform.

In programs processing large data sizes errors relating to indexing large arrays or eternal loops may occur. The following example contains 2 errors:
const size_t size = ...; char *array = ...; char *end = array + size; for (unsigned i = 0; i != size; ++i) { const int one = 1; end[-i - one] = 0; } |
The first error consists in that if the size of the data being processed excesses 4 GB (0xFFFFFFFF) an eternal loop may occur as 'i' variable has 'unsigned' type and will never reach 0xFFFFFFFF value. I write deliberately that it can occur but not necessarily. It depends on what code the compiler will build. For example, in debug mode the eternal loop will be present and in release-code there will be no loop as the compiler will decide to optimize the code using a 64-bit register for the counter and the loop will be correct. All this adds much confusion and the code which worked yesterday can fail to work today.
The second error relates to parsing the array from beginning to end for what negative indexes' values are used. This code will operate well in 32-bit mode but when executed on a 64-bit computer access outside the array's limits will occur at the first iteration of the loop, and there will be a program crash. Let's study the reason of such a behavior.
According to C++ rules "-i - one" expression on a 32-bit system will be calculated as follows: (at the first step i = 0):
"-i" expression has unsigned type and has 0x00000000u value.
'one' variable will be extended from 'int' type to unsigned type and will equal 0x00000001u. Note: int type is extended (according to C++ standard) up to 'unsigned' type if it participates in an operation where the second argument has unsigned type.
A subtraction operation takes place in which two values of unsigned type participate and the result of the operation equals 0x00000000u - 0x00000001u = 0xFFFFFFFFu. Note that the result will have unsigned type.
On a 32-bit system access to the array by the index 0xFFFFFFFFu is the same as using -1 index. That is end[0xFFFFFFFFu] is an analog of end[-1]. As a result the array's items will be processed correctly.
In a 64-bit system the situation will be quite different concerning the last point. Unsigned type will be extended to signed ptfdiff_t type and the array's index will equal 0x00000000FFFFFFFFi64. As a result an overflow will occur.
To correct the code you should use ptrdiff_t and size_t types.
There are errors which are nobody's fault but they are still errors. Imagine that long long ago in a faraway galaxy (in Visual Studio 6.0) a project was developed which contained CSampleApp class - a successor of CWinApp. In the basic class there is a virtual function WinHelp. The successor overlaps this function and performs all the necessary actions. This process is shown in Figure 7.

After that the project is ported on Visual Studio 2005 where the prototype of WinHelp function has changed but nobody will notice it because in 32-bit mode DWORD and DWORD_PTR types coincide and the program continues operating correctly (Figure 8).

The error is waiting to occur on a 64-bit system where the sizes of DWORD and DWORD_PTR types are different (Figure 9). In 64-bit mode classes appear to contain two DIFFERENT functions WinHelp and that is of course incorrect. Keep in mind that such traps may hide not only in MFC where some functions have changed their arguments' types but in the code of your applications and third-party libraries as well.

There are a lot of examples of such 64-bit errors. Those who are interested in this topic and would like to know more about these errors see the article "20 issues of porting C++ code on the 64-bit platform" [12].
As you see the stage of searching hidden errors is a nontrivial task, and besides, many of them will occur irregularly and only at large input data. Static code analyzers are good for diagnosing such errors as they can check the whole code of an application independently from the input data and the frequency of its sections' execution in real conditions. There is sense in using static analysis both at the stage of porting an application on 64-bit platforms to find most errors at the very beginning and in further development of 64-bit solutions. Static analysis will warn and teach a programmer to better understand the peculiarities of errors relating to a 64-bit architecture and to write more efficient code. The author of the article is a developer of one of such specialized code analyzers named Viva64 [13]. To learn more about the tool and to download a demo version visit the site of OOO "Program Verification Systems" company.
For justice' sake we should say that Gimpel PC-Lint and Parasoft C++test code analyzers have sets of rules for diagnosing 64-bit errors. But, firstly, these are general-purpose analyzers and the rules of diagnosing 64-bit errors are incomplete. Secondly, they are intended mostly for LP64 data model used in the family of Linux operation system and that's why they are not so useful for Windows programs where LLP64 data model is used [14].
The step of searching errors in program code described in the previous section is necessary but insufficient. None of the methods, including static code analysis, can guarantee detection of all the errors, and the best result can be achieved only when combining different methods.
If your 64-bit program processes a larger data size than the 32-bit version, you need to extend tests to include processing data with the size more than 4 GB. This is the border beyond which many 64-bit errors begin to occur. Such tests may take much more time and you must be prepared for it. Usually tests are written in such a way that each test could process a small number of items and thus make it possible to perform all the internal unit-tests in several minutes while automatic tests (for example, using AutomatedQA TestComplete) could be performed in several hours. It is nearly absolutely certain that the sorting function sorting 100 items will behave correctly at 100000 items on a 32-bit system. But the same function can fail on a 64-bit system while trying to process 5 billion items. The speed of executing a unit-test can fall in million times. Don't forget about the cost of adapting tests while mastering 64-bit systems. A good solution is to divide unit-tests into quick (working with small memory sizes) and slow ones processing gigabytes and executed, for example, in the nighttime. Automated testing of resource-intensive 64-bit programs can be organized on the basis of distributed calculations.
There is one more unpleasant thing. You will hardly succeed in using tools like BoundsChecker for searching errors in resource-intensive 64-bit programs consuming large memory size. The reason is a great slowdown of the programs being tested what makes this approach very inconvenient. In the mode of diagnosing all the errors relating to memory operation, Parallel Inspector tool included into Intel Parallel Studio will slow down execution of an application in 100 times on the average (Figure 10). It is very likely that you will have to leave the algorithm being tested for the night to see the results only the next day while normally this algorithm operates just 10 minutes. And still I'm sure that Parallel Inspector is one of the most useful and convenient tools when working in the mode of searching memory-operation errors. You just must be ready to change the practice of error diagnosing and keep it in mind when planning to master 64-bit systems.

And the last thing. Don't forget to add tests checking compatibility of data formats between the 32-bit and 64-bit versions. Data compatibility is often violated during migration because of writing of such types as size_t or long (in Linux systems) into files.