The pen-based interface, called Notate, allows users of computational, digital notebooks to open drawing canvases and handwrite diagrams within lines of traditional, digitised computer code.

This innovation allows far more creativity for programmers, who would usually have to type all of the commands into the code. 

Notate is powered by a deep learning model that has been designed to allow programmers to link handwritten notes and diagrams to textual code and vice versa. For instance, it can identify handwritten programming symbols, like "n", and link them up to their typewritten equivalents.

Programming tool turns handwriting into computer code / Cornell University

Image credit: The 35th Annual ACM Symposium on User Interface Software and Technology (2022)

"A...