How to design new driver for my own board

you can see here that even best fine tuned usb devices on RT kernel has a round trip close to 10ms
interfacinglinux.com/linux-compatible-audio-interfaces/
compared to stable 0.1-0.5ms with Ethernet or SPI