Package unsafe is in the spec, but converting unsafe.Pointer to uintptr (which is how I'm supposing you'd do your pointer arithmetic) is implementation-defined.
This means I could create a perfectly legal implementation of Go where such things result in complete nonsense. I don't think package unsafe "counts".
If package unsafe doesn't "count", anything that C compilers allow beyond what ANSI C specifies don't count as C, thus many of its system programming features just vanish.
https://play.golang.org/p/m3OdaXH98_