According to how much money you have to spend, there are various ways to start coding:
The Yaroze comes with a version of gcc/as, built to run under DOS/Windows and targetted for a little endian MIPS processor. The version of ld supplied knows how to create PSX 'EXE' files that can be downloaded and run on the playstation.
The library used for the Yaroze is also supplied in the form of an EXE, 'libps.exe'. This is downloaded to the playstation prior to running your software, and sits in the lowest 400K or so of memory. To make use of this library you link with 'libps.a'; this is a set of stubs that simply resolves calls to library routines into jumps to the appropriate address in libps.exe.
In order for anyone else to use a Yaroze program you have written therefore, then will need to have a copy of libps.exe. This is not too much of a problem though as any Playstation CD that contains a Yaroze game (such as most of them on the front of Playstation Magazines nowadays) will include a copy of libps.exe.
The PsyQ tools are basically the professional development compilers, assemblers and libraries for the Playstation. These are available to buy, and should allow people who were previously relying on gcc to start to make more extensive use of the features of their PSXs.
These libraries have several advantages over the Yaroze libs in that they are not only much more powerful, but also are supplied in a form which links properly with your code; this means that any program you write will only have the library functions it uses linked with it - not the whole library.
But what if you don't have a Yaroze, and you can't afford the professional tools? Well, there is a free version of gcc/as targetted for little endian MIPS processors and a free version of the gnu linker that will create PSX EXEs. In theory you can just get the GCC source and build them yourselves, but thats a very daunting task for most people. Fortunately the 'Not Yaroze' page has versions to download, and I reproduce them here with english ReadMe files rather than the original Japanese! These have been compiled so as they run in a DOS window under Windows 95 (not on earlier Windows or DOS versions!) and output code suitable for sending to the Playstation with either Caetla/Comms kit or Xplorers.
All this leaves is the problem of the libraries; as mentioned before libps.exe is available on most cover discs, but you still need libps.a and the C header file for it. Public Domain versions of these have been made and are reproduced here.
Some useful code fragments (too small to properly be called libraries) are available to download in the releases section.