Conditional Parsing and Preprocessor Support
Conditional Parsing and Preprocessor Support
This section discusses features and issues that apply mainly to C/C++ and C# language files that use a preprocessor and have
#if or
#ifdef
statements.
Working with Inactive Code - ifdef Support
Source Insight can recognize inactive blocks of code that are disabled at compile-time with
#ifdef
,
#if
, and
#elif
directives in languages that support those statements. There are project-specific and global lists of conditional identifier values. To edit the conditional constant values, select Options > Preferences: Language
and click on either "Project Specific Conditions", or "Global Conditions". See “Conditional Parsing List” on page 171. You can also edit the conditions using the Edit Condition command on the right-click menu of a
source window. See “Edit Condition” on page 195.
Note:
Conditional statements such as
#if and #ifdef are only interpreted when all the condition values are specified in the
Edit Conditions
dialog box.
By default, Source Insight ignores the conditional directives altogether. It attempts to make sense of all branches in a conditional compilation construct. Often, this works well because declarations in the conditional branches do not interfere with each other.
However, sometimes a tricky declaration may be broken in the middle with an
#ifdef
. This will often confuse
Source Insight. For example: void DoThing(
int param1,
#ifdef ABC
int param2)
#else
int param2, param3)
#endif
If you are not interested in code that is inactive, you can specify condition values. See “Conditional Parsing” on page 58.
Inactive code is formatted with the Inactive Code style.
Blocks of code that are inactive are displayed in the "Inactive Code" style. For example:
Figure 3.2 The "Inactive Code" style is displayed.
Conditional Parsing
Conditional parsing applies only to languages that support conditional compilation in Source Insight: C/C++,
C#, etc. See “Edit Condition” on page 195.
Source Insight maintains two types of condition variable lists.
•
Global condition list
. This applies to all projects. This list is saved in the configuration file, which contains your customizations.
•
Project-Specific condition list
. This list is saved with each project. You can have different condition variables defined for each individual project. For example, you could have "RELEASE" defined in one project, and "DEBUG" defined in another.
58 Source Insight 4 User Guide
Conditional Parsing and Preprocessor Support
The two condition lists are combined when Source Insight parses a file. The project-specific conditions take precedence over the global conditions.
Condition Variables
Condition variables can be used in expressions in #if, #ifdef, #ifndef, and #elif statements. These are typically constant values defined in a header file, or on the compiler command line. For example:
#if VER < 3 && DEF_OPEN != 0
....
In this example, two condition variables are used: VER and DEF_OPEN. Each variable value can be specified
using the Edit Condition command. See “Edit Condition” on page 195.
Each condition variable can have any textual value. As in C and C++, any numerical value that equals zero is considered "False", and any non-zero value is "True".
Ignoring Condition Variables
If you do not specify a variable's value, then any preprocessor statement that includes that variable is skipped and simply ignored. This is the default behavior for any #if-type statement in Source Insight. If the condition is not defined using the Edit Condition dialog, then Source Insight ignore the conditional and all branches of the #if-#else block are parsed by Source Insight.
For example:
#if VER < 3 && WINVER >= 5
int a = 1;
#else
int a = 2;
#endif
If both VER and WINVER are defined using Edit Condition, then the expression in the #if statement will be evaluated, and only one of the branches will be active. However, if either of those variables are not defined in
Source Insight, then both branches will be active.
Editing the Condition Variables
To edit the value of a conditional variable, right-click on it and select Edit Condition. See “Edit Condition” on page 195. When you edit the condition list, Source Insight will ask you if you want to re-parse your whole
project. You should make all your changes to the condition list first, and then re-parse your whole project.
Until your project is re-parsed, the symbol information stored in Source Insight's symbol database will not reflect the changes you made.
Source Insight 4 User Guide 59
Conditional Parsing and Preprocessor Support
Parsing Considerations and Parsing Problems
The language parsing in Source Insight is somewhere between a strict grammar parser, and a simple pattern matcher. This is good because the parser is error-tolerant and works well on code that can be incomplete or have some syntax errors. However, there are some points to keep in mind:
•
The source files can have syntax errors. They don't have to be compiled. That means symbols will be found in source code that doesn't even compile or is in an intermediate state, which is most of the time!
•
Preprocessor macros are not expanded before Source Insight parses the code. This may sound bad, but in reality it works quite well. Source Insight parses your program at its most abstract level - at the same level as it was written. Among other things, this also allows call trees to contain function-like macros.
•
Symbol declarations are parsed from all of the source code, not just what was active at the time of compilation. For example, C/C++ code inside of #ifdef-#endif clauses are also added to the symbol database, even if the #ifdef branch is not active when you compile. This can be a great help if you are working on a multi-state program, and you need to be aware of all cases, not only what the compiler sees. If you do want to omit inactive code blocks, you can define condition values with the Edit Condi-
tion
command, or in Options > Preferences: Languages.
•
The text in comments and constants is also indexed for searching. Source Insight is focused on source level code; not just what the compiler transforms into object code.
•
All header files are assumed accessible in all source files. Source Insight does not notice what header files are included in each particular source file. Therefore, all symbols are known at all points. This is technically inaccurate with respect to how a compiler will see your program, but in most cases it works well. This is a trade-off of speed versus complete correctness.
•
Some programming styles may cause symbols not to be found by Source Insight. The default parsing that Source Insight uses works very well with most programming styles. In the event that you have some declarations types that Source Insight can't recognize, you can add Token Macros, which expand
during a preprocessing phase, or you can add a custom parsing regular expression pattern. See “Preprocessor Token Macros” on page 62.
Parsing Limitations
Because regular C/C++ preprocessing is not performed by Source Insight's parsers, some coding styles affect parsing correctness in Source Insight.
Having
#ifdef
blocks break up an individual declaration will confuse Source Insight. If it cannot be avoided, and Source Insight doesn't parse the code correctly, you will need to define the condition value using Edit
Condition
, so that Source Insight will disable the inactive block of code causing the confusion. See “Condi-
tional Parsing” on page 58.
For example: void MyFunc
#ifdef XYZ
(int param1, int param2)
#else
(long param1, long param2)
#endif
{
...
}
Replacing standard language keywords or combinations with #define'd substitutions will confuse Source
Insight. If you cannot avoid this, then you will need to define Token Macros to support them. See “Preprocessor Token Macros” on page 62.
60 Source Insight 4 User Guide
Conditional Parsing and Preprocessor Support
For example:
#define ourpublic public class D : ourpublic B { … }
This causes a problem because Source Insight doesn't know that the keyword ourpublic
really means public
.
Fixing "Parse-Too-Complex"
If Source Insight encounters too many problems trying to parse a file, the symbol window at the left side of the source window will say "Parse Too Complex" instead of listing the symbols in the file.
The Parse-too-complex can often be handled by using the Edit Condition dialog, or defining Source Insight preprocessor macros (token macros).
The Parse-too-complex usually happens in the following situations.
#ifdef Statements Confuse the Parser
In this situation, you have some
#ifdefs
that confuse the parser, especially when it splits the declaration of some symbol. This can be handled by using the Edit Condition command to define the state of the
#if
or
#ifdef
condition variable. See “Edit Condition” on page 195.
For example:
#ifdef ABC typedef struct FOO_V1 {
#else typedef struct FOO_V2 {
#endif int x;
} FOO;
In the above example, the declaration of the typedef-struct is split up with
#ifdef
-
#else
-
#endif
. Source
Insight tries to parse this by ignoring the preprocessor statements because the conditional value
ABC
is not defined using Edit Condition. Source Insight ends up interpreted the code like this: typedef struct FOO_V1 { typedef struct FOO_V2 { int x;
} FOO;
This is obviously not valid code, and therefore Source Insight cannot parse it. However, if you use Edit Condi-
tion
to define the value of
ABC
then Source Insight will parse the code.
Declaring Things Using a Preprocessor Macro
In this situation, you are using a preprocessor macro to declare things. An example of this might be something like this:
#define MyArray(name) char *name[]
MyArray(foo) =
{
"abc",
"xyz",
...
};
In this case, Source Insight thinks
MyArray(foo)
is a function, so it skips ahead and eventually gets confused.
This can be fixed usually by defining a token macro in the c.tom file located in your
Documents\Source
Insight 4.0
folder. See “Preprocessor Token Macros” on page 62. To fix this example, you need to define a
token macro for
MyArray
.
Source Insight 4 User Guide 61
Conditional Parsing and Preprocessor Support
Replacing Keywords, Operators, or Delimiters with Preprocessor Macros
In this situation, a preprocessor macro contains syntax keywords or braces.
#define DeclareMyFunction(name) void name() {
DeclareMyFunction(foo)
..body of function
}
This fails to parse because Source Insight does not expand the macro and fails to see the function name or the opening brace. Again, this can be fixed by defining a token macro in the c.tom file located in your
Documents\Source Insight 4.0
folder. See “Preprocessor Token Macros” on page 62. To fix this example, you
need to define a token macro for DeclareMyFunction.
Preprocessor Token Macros
Source Insight contains its own preprocessor, which is used when parsing files. The preprocessor macros are called Token Macros. They are token substitutions that occur as Source Insight parses a file. They allow
Source Insight to handle any special language keywords not known to Source Insight's parsers, and to handle special C/C++ preprocessor substitutions that would otherwise confuse it.
Preprocessor macros are normally not expanded by Source Insight
Source Insight does not expand C/C++ preprocessor macros when it parses your files. Because of this, certain preprocessor macros and constants can fool Source Insight. Therefore, token macros are used to let Source
Insight selectively expand some preprocessor substitutions.
For example, the STDMETHOD(methodname) macro is used to declare a COM method function. Source
Insight ships with a default C/C++ token macro file (c.tom) that has entries for this macro, and many others.
Token Macro Files
The token macros are listed in a file with a
.tom
extension. The global token macro file resides in the Source
Insight program directory. The project-specific token macro file, if any, is stored in project's data directory.
The project token macro file is combined with the global file, with the project macros taking precedence.
Token Macro Syntax
A token macro file consists of token macros, one per line. The format of a token macro is: macroname <no text here means macro is a no-op> macroname substituted text here macroname(parameter list) substituted text with parameter names macroname(parameter) text##parameter // concatenates text
; comments begin with a semicolon
Some examples of token macros:
MyStructure(sname) struct sname
NoOperation
BuildName(name1, name2) name1##name2
Each built-in language parser has a corresponding token macro file. The name of the token macro file for each language is summarized below:
Language
C/C++/Objective-C
Java
Table 3.1: Token Macro Files for Different Languages
Resource Files
File Name
C.tom – a default copy ships with
Source Insight.
Java.tom
Rc.tom
62 Source Insight 4 User Guide
Conditional Parsing and Preprocessor Support
Table 3.1: Token Macro Files for Different Languages
Language
x86 Assembly Language
Perl
File Name
X86.tom
Perl.tom
Editing Token Macros
If you want to change the token macros, simply open the token macro file, make your changes, and save the file. Source Insight will recognize that the token macros have changed for the appropriate language. Open files are automatically re-parsed.
Save a token macro file to get Source Insight to recognize the macros
When you edit a token macro file, you must save it to disk before Source Insight will re-parse your open files.
However, Source Insight will not automatically re-parse your whole project. You should make all your changes to the token macro file first, then use the Rebuild Project command to re-parse your whole project.
Until your project is re-parsed, the symbol information stored in Source Insight's symbol database will not reflect the changes you made to your token macros.
Project Specific Token Macros
Each project can have its own set of token macro files. Source Insight does not create them automatically, but you can yourself. A project token macro file is saved in the project's data directory. When Source Insight parses a source file, it combines the project token macros with the global set saved in the Source Insight program directory. The project token macros take precedence over the global ones. By adding project specific token macros, you can tailor the token macro expansion for each project individually.
Source Insight 4 User Guide 63