regular expression patterns for c syntax none of these patterns has any capturing however a number of them have capturing markers compatible with utils setcapturegroups general character literal string literal end string literal keyword textwrap dedentr b auto extern register static threadlocal typedef const volatile signed unsigned char short int long float double void struct union enum goto return sizeof break continue if else for do while switch case default entry b keyword end keyword keyword joinkeyword split identifier r azazazaz09 we use a negative lookahead to filter out keywords strictidentifier rf keyword b identifier b anonidentifier rf keyword b identifier d b types simpletype textwrap dedentrf simple type b void signed unsigned implies int signed unsigned s long short s char short int long float double b end simple type variable declarations type spec reference to a compound type reference to a typedef end type spec declarator textwrap dedentrf declarator possibly abstract ptrqualifier s identifier strictidentifier inside the brackets is actually a constant expression s s s arrays s wrappedidentifier strictidentifier inside the brackets is actually a constant expression s s s arrays s func ptr s ptrqualifier s funcidentifier strictidentifier inside the brackets is actually a constant expression s s s arrays s we allow for a single level of paren nesting in parameters s end declarator var decl and typedef and func return type end var decl initializer textwrap dedentrf initializer no nested parens e g func ptr s a string literal indstringliteral 4 s indstringliteral 4 a simple initializer indstringliteral 4 a structarray literal we only expect compound initializers with singlevariable declarations indstringliteral 5 s note this lookahead end initializer compound type declarations inline compound type decl typed member technically it doesn t have to have a type if it doesn t have a declarator then it will have a size and vice versa sized member this is actually a constant expression enummemberdecl textwrap dedentrf s close s name identifier s s init this is actually a constant expression indstringliteral 4 s ending statements simple statement body note this lookahead end simple statement body simplestmt textwrap dedentrf simple statement simplestmt stmtinline initializer return b s indinitializer 5 variable assignment s strictidentifier s s strictidentifier s s d s s s indinitializer 4 catchall return statement return b indstringliteral 6 s simple statement indsimplestmtbody 4 s simpleending end simple statement compound statement end compound statement function bodies local textwrap dedentrf an empty statement empty inline type decl inlineleading s inlinepre storageclass s typequalifier s inlinepre inlinekind compoundtypekind s inlinename strictidentifier s var decl storage storageclass storage s vardecl indvardecl 5 initializer we expect only basic initializers s s varinit indinitializer 6 s varending indcompoundstmt 2 startofblock blockleading indstringliteral 5 presumably we will not see s s blockleading blockopen indsimplestmt 2 endofblock blockclose inline type decl var decl we only look for static variables everything else global declarations global textwrap dedentrf an empty statement empty compound type decl maybe inline compoundleading s compoundkind compoundtypekind s compoundname strictidentifier s bogus inline decl artifact this simplifies resolving the relative syntactic ambiguity of inline structs forwardkind compoundtypekind s forwardname anonidentifier maybeinlineactual typedef b typedef b s typedefdecl indvardecl 4 we expect no inline type definitions in the parameters s s typedeffuncparams s s func decldefinition var decls xxx dedicated pattern for funcs more restricted varstorage storageclass s funcinline b inline b s vardecl indvardecl 4 func decl definition we expect no inline type definitions in the parameters s s funcparams s s funcdelim this is some oldschool syntax s s we throw away the bare names strictidentifier s s strictidentifier s s we keep the trailing param declarations funclegacyparams there s at least one typequalifier s indtypespec 7 s inddeclarator 7 s s typequalifier s indtypespec 8 s inddeclarator 8 s s var typedef initializer we expect only basic initializers s s varinit indinitializer 6 s varending regular expression patterns for c syntax none of these patterns has any capturing however a number of them have capturing markers compatible with utils set_capture_groups general character literal x hex hex 0 d d o 01 d d o2 0 4 d o25 0 5 string literal end string literal b auto extern register static _thread_local typedef const volatile signed unsigned char short int long float double void struct union enum goto return sizeof break continue if else for do while switch case default entry b keyword _keyword end keyword we use a negative lookahead to filter out keywords types simple type b void signed unsigned implies int signed unsigned s long short s char short int long float double b end simple type variable declarations type spec _ind simple_type 2 _ typeof _ s s s strict_identifier s reference to a compound type compound_type_kind s anon_identifier reference to a typedef strict_identifier end type spec declarator possibly abstract ptr_qualifier s identifier strict_identifier inside the brackets is actually a constant expression s s s arrays s wrapped_identifier strict_identifier inside the brackets is actually a constant expression s s s arrays s func ptr s ptr_qualifier s func_identifier strict_identifier inside the brackets is actually a constant expression s s s arrays s we allow for a single level of paren nesting in parameters s end declarator var decl and typedef and func return type storage storage_class s type_qual type_qualifier s type_spec _ind type_spec 4 s declarator _ind declarator 4 end var decl initializer no nested parens e g func ptr s a string literal _ind string_literal 4 s _ind string_literal 4 a simple initializer _ind string_literal 4 a struct array literal we only expect compound initializers with single variable declarations _ind string_literal 5 s note this lookahead end initializer compound type declarations inline compound type decl compound_type_kind compound_type_kind s compound_type_name strict_identifier s typed member technically it doesn t have to have a type specifier_qualifier type_qualifier s _ind type_spec 5 if it doesn t have a declarator then it will have a size and vice versa s declarator _ind declarator 6 sized member s s size this is actually a constant expression d s ending s close s close s name identifier s s init this is actually a constant expression _ind string_literal 4 s ending statements simple statement body _ind string_literal 3 note this lookahead end simple statement body simple statement simple_stmt stmt inline initializer return b s _ind initializer 5 variable assignment s strict_identifier s s strict_identifier s s d s s s _ind initializer 4 catchall return statement return b _ind string_literal 6 s simple statement _ind simple_stmt_body 4 s simple_ending end simple statement compound statement b compound_bare else do b compound_labeled case b _ind string_literal 7 s default strict_identifier s compound_paren for while if switch s note this lookahead s end compound statement function bodies an empty statement empty inline type decl inline_leading s inline_pre storage_class s type_qualifier s inline_pre inline_kind compound_type_kind s inline_name strict_identifier s var decl storage storage_class storage s var_decl _ind var_decl 5 initializer we expect only basic initializers s s var_init _ind initializer 6 s var_ending _ind compound_stmt 2 start of block block_leading _ind string_literal 5 presumably we will not see s s block_leading block_open _ind simple_stmt 2 end of block block_close inline type decl inline_leading s inline_pre storage_class s type_qualifier s inline_kind compound_type_kind s inline_name strict_identifier s var decl we only look for static variables static_decl static b s type_qualifier s _ind type_spec 4 s _ind declarator 4 s static_init s _ind initializer 4 s static_ending everything else delim_leading _ind string_literal 4 s block_open block_close stmt_end global declarations an empty statement empty compound type decl maybe inline compound_leading s compound_kind compound_type_kind s compound_name strict_identifier s bogus inline decl artifact this simplifies resolving the relative syntactic ambiguity of inline structs forward_kind compound_type_kind s forward_name anon_identifier maybe_inline_actual typedef b typedef b s typedef_decl _ind var_decl 4 we expect no inline type definitions in the parameters s s typedef_func_params s s func decl definition var decls xxx dedicated pattern for funcs more restricted var_storage storage_class s func_inline b inline b s var_decl _ind var_decl 4 func decl definition we expect no inline type definitions in the parameters s s func_params s s func_delim this is some old school syntax s s we throw away the bare names strict_identifier s s strict_identifier s s we keep the trailing param declarations func_legacy_params there s at least one type_qualifier s _ind type_spec 7 s _ind declarator 7 s s type_qualifier s _ind type_spec 8 s _ind declarator 8 s s var typedef initializer we expect only basic initializers s s var_init _ind initializer 6 s var_ending
import textwrap


def _ind(text, level=1, edges='both'):
    indent = '    ' * level
    text = textwrap.indent(text, indent)
    if edges == 'pre' or edges == 'both':
        text = '\n' + indent + text.lstrip()
    if edges == 'post' or edges == 'both':
        text = text.rstrip() + '\n' + '    ' * (level - 1)
    return text





HEX = r'(?: [0-9a-zA-Z] )'

STRING_LITERAL = textwrap.dedent(rf)

_KEYWORD = textwrap.dedent(r)
KEYWORD = rf
_KEYWORD = ''.join(_KEYWORD.split())

IDENTIFIER = r'(?: [a-zA-Z_][a-zA-Z0-9_]* )'

STRICT_IDENTIFIER = rf'(?: (?! {_KEYWORD} ) \b {IDENTIFIER} \b )'
ANON_IDENTIFIER = rf'(?: (?! {_KEYWORD} ) \b {IDENTIFIER} (?: - \d+ )? \b )'





SIMPLE_TYPE = textwrap.dedent(rf)

COMPOUND_TYPE_KIND = r'(?: \b (?: struct | union | enum ) \b )'





_STORAGE = 'auto register static extern _Thread_local'.split()
STORAGE_CLASS = rf'(?: \b (?: {" | ".join(_STORAGE)} ) \b )'
TYPE_QUALIFIER = r'(?: \b (?: const | volatile ) \b )'
PTR_QUALIFIER = rf'(?: [*] (?: \s* {TYPE_QUALIFIER} )? )'

TYPE_SPEC = textwrap.dedent(rf)

DECLARATOR = textwrap.dedent(rf)

VAR_DECL = textwrap.dedent(rf)

INITIALIZER = textwrap.dedent(rf)





STRUCT_MEMBER_DECL = textwrap.dedent(rf)

ENUM_MEMBER_DECL = textwrap.dedent(rf)





SIMPLE_STMT_BODY = textwrap.dedent(rf)
SIMPLE_STMT = textwrap.dedent(rf)
COMPOUND_STMT = textwrap.dedent(rf)





LOCAL = textwrap.dedent(rf)

LOCAL_STATICS = textwrap.dedent(rf)





GLOBAL = textwrap.dedent(rf)