Cast insertion fails when cast needs to be placed inside a macro

[john-h-kastner]

In the following example, 3C makes the type of b in foo _Ptr<int> because it is used safely inside the function. The function is called from inside the macro BAR with an argument that cannot be rewritten to a checked type (because the argument is declared in the same macro). 3C want to insert a cast from int* to _Ptr<int> to make the call correct, but it cannot do this inside the macro.

#define BAR \
  void bar(int *a) {\
    foo(a);\
  }

void foo(int *b) {}
BAR

After the changes in PR #434 a warning is emitted indicating where the rewriting fails. and suggesting what cast should be inserted.

/home/cc/checkedc-examples/cast_fail.c:7:1: warning: Unable to surround expression with cast.
Intended cast: "_Assume_bounds_cast<_Ptr<int>>()"
FOO
^~~
/home/cc/checkedc-examples/cast_fail.c:3:9: note: expanded from macro 'FOO'
    foo(a);\
        ^

Since programs should compile without manual fixes when compiling without --alltypes, 3C should rewrite the code so that this cast does not need to be placed. For instance, 3C could notice that foo is called from an non-rewritable source range, and based on this information constrain the functions parameter types equal to the type of any arguments.
The fix for this issue should be verified against Lua.

[mattmccutchen-cci]

Another possible solution (which we could consider offering as another option, or the only option if it's too difficult to generate the correct constraints) is to inline the macro expansion and then rewrite the middle of it (roughly as I described in the first long paragraph of #400 (comment)). After my work on the #434 enhancement, I have a general idea of how to approach this using the Clang API, though it may be a heavy lift. We could wait and see if any better APIs become available after the Clang 11 upgrade.

[john-h-kastner]

Mostly fixed in #491, but there's still a non-critical cast insertion fail in lua:

lobject.c:230:29: warning: Unable to surround expression with cast.
Intended cast: "((char **))"
  *result = (mode == 'x') ? lua_strx2number(s, &endptr)  /* try to convert */
                            ^~~~~~~~~~~~~~~~~~~~~~~~~~~
./luaconf.h:613:31: note: expanded from macro 'lua_strx2number'
#define lua_strx2number(s,p)            lua_str2number(s,p)
                                        ^
./luaconf.h:480:41: note: expanded from macro 'lua_str2number'
#define lua_str2number(s,p)     strtod((s), (p))
                                            ^
lobject.c:231:29: warning: Unable to surround expression with cast.
Intended cast: "((char **))"
                          : lua_str2number(s, &endptr);
                            ^~~~~~~~~~~~~~~~~~~~~~~~~~
./luaconf.h:480:41: note: expanded from macro 'lua_str2number'
#define lua_str2number(s,p)     strtod((s), (p))

---

This demonstrates what's triggering the error

void baz(int **x : itype(_Ptr<_Ptr<int>>));
#define bar(a) baz(&a)
void foo() {
  int *a = 1;
  bar(a);
}

Since baz doesn't have body, we can't have it solve to an unchecked type. Instead we try to insert a cast, and this fails.

[john-h-kastner]

Re-opened because of above comment. This is not quite fully resolved.