Since version 1.80 or so, the kernel implements kernel address space layout randomization to discourage ROP attacks.
Since version 1.80 or so, the kernel makes use of stack canaries to detect stack buffer overflows and halts the system when an overflow is detected.
Memory domains is a feature in ARM MMU that provides an easy way of showing and hiding groups of addresses as well as their permissions. When a syscall is made, the handler disables all access to memory domains for user memory so kernel code cannot directly access user memory. This means if a user pointer is passed in and the kernel forgets to check it and dereferences it directly, it will abort. In order to access user memory, special functions are used that temporarily enables all domains and the access is implemented with the ARM unprivileged access instructions
STRT to make sure the access functions cannot read or write in kernel memory space. As long as the domain disable code in the syscall hander is secure and the user memory access functions are secure, there is no need for additional checks implemented per function. Additionally all non-code pages are marked as "execute never" in both kernel and userland.
Usermode stack pivoting protection
Since unknown version (seen on 3.18) the kernel will terminate an application if it notices that its stack pointer register is not pointing into the stack memory.
User&kernel heap overflow protection
dlmalloc, used for heap allocations, is compiled with -DFOOTERS=1 to enable more heap overflow checks. Additionally, a custom SceNetPs malloc implementation also does some heap overflow checks on its own.
List of kernel libraries
For a list of all kernel libraries, check out Libraries.