Communication with the Filer: the Data Transfer Protocol

These protocols are understood by the Filer, and by applications that load and save files. In fact, the Filer is not a particularly special case. The following cases are considered:

(1) The user drags an icon from an application to a Filer window. The application saves the file.

(2) The user drags an icon from an application to a window of another application. The transfer is achieved via a temporary file.

(3) The user drags an icon from an application to a window of another application. The transfer is achieved (if both sides agree) using direct transfer of memory between the two address spaces.

(4) The user drags an icon from the Filer to the window of an application. The application inserts or loads the file.

(5) The user double-clicks on an icon in the Filer. An existing application opens the file.

There are other cases which generalise some of these, with an application "pretending" to be the Filer. For instance, when a message arrives for MailMan, it simulates case (5) to cause the editor to display the incoming message.

This section assumes knowledge of the Wimp program interface. It is separate because the facilities are actually provided by the Filer rather than the Wimp. Careful study of the sections on message passing is recommended.

The protocols do not specify whether event codes 17 or 18 are used within the protocol, as either is acceptable at any stage of the protocols. Type 18 messages are needed when confirmation of non-reply is required, which is provided by the Wimp. Type 18 messages are recommended when a reply is expected.

Also, messages may include excess junk at the end of the form. The precise size of the message is not part of the protocol.

Some of the message formats have unused or copied portions. This is to make it easier to reply to certain messages by overwriting some fields in the incoming message and then just sending it back. Note, however, that these are essential parts of the protocol and must be copied precisely in order to adhere to the protocol. In some cases acknowledgements and replies are not used, this makes it especially important to stick to the letter of this document (rather than just "hacking until it works with the Filer") to prevent surprises when programs from different sources are brought together.

The following message is sent by an application wishing to save a document (or a selected part of a document):

1 DataSave

   R0     17 (usually)
R1!0 size
     !12  0
     !16  1 ; DataSave
     !20  destination window handle             ;
     !24  destination window icon               ; copied from
     !28  destination x coord (screen coords)   ; Wimp_GetPointerInfo
     !32  destination y coord (screen coords)   ;
     !36  estimated size of data, in bytes
     !40  file type of data
     !44  proposed leaf-name of file, 0-terminated

The destination window handle, icon and coordinates are those generated by Wimp_GetPointerInfo at the end of a drag.

The file type word is a value in the range 0..&fff for typed files, or one of the following:
&1000 for a directory
&2000 for an application directory &3000 for a load/exec file

If the target window is a directory viewer (or any other window which wishes the information to be written only to a file) then it replies as follows:

2 DataSaveAck (save data to file)

   R0     17 (usually)
R1!0 size
     !12  my_ref field of DataSave message
     !16  2  ; save data to here
     !20..40 ; left unchanged from relevant DataSave
     !44  full path-name of file, 0-terminated

Note that the type and size information are typically ignored when generating this response. The Filer implements this form of reply for directory viewer windows.

An editor, on receiving this reply, should save the data specified by the user in the named file. If this save fails, report using Wimp_ReportError and the transaction stops.

The editor should also mark the document as not modified, if the whole document is being saved (rather than some selected portion). An exception to this is that if the "estimated size" field of the DataSaveAck is negative then do not mark the document as unmodified: this is a signal from the recipient that this file is a temporary one, rather than a good repository for future versions of the file. When marking the document as unmodified, also remember the full path-name so that future saves can be performed without the user having to drag icons.

The editor then issues this acknowledgement:

3 DataLoad (drag file from Filer / I have saved data to a file)

   R0     18 (usually)
R1!0 size
     !12  my_ref field of DataSaveAck message (or 0 if from Filer)
     !16  3 ; load data from here
     !20  destination window handle             ;
     !24  destination icon handle               ; copied from
     !28  destination x coord (screen coords)   ; Wimp_GetPointerInfo
     !32  destination y coord (screen coords)   ;
     !36  must be <= 0 ; size field in other messages
     !40  file type
     !44  full path-name of file, 0-terminated

(This acknowledgement is not used by the Filer, but is used in cases where the file is a temporary file used to communicate between two applications).

This message is also used in an entirely independent context. The Filer sends this message when a file has been dragged into a window belonging to another application. The application is then free to copy or insert the file, if it so desires. If it does so successfully it should reply as follows:

4 DataLoadAck

   R0     17 (usually)
R1!0 size
     !12  my_ref field of DataLoad message
     !16  4 ; DataLoadAck

This message is only sent if the load was successful.

Thus, the case of a file being saved to the Filer in fact involves 4 messages being sent:

(1)  DataSave        (application to Filer)
(2)  DataSaveAck     (Filer back to application)
                     (application saves the file)
(3)  DataLoad        (application to Filer, as acknowlegement)
                     (the Filer just turns this around, doing nothing)
(4)  DataLoadAck     (Filer to application)
                     (if this is not received, the application generates an
                     error "Bad Data Transfer, Receiver Dead" and
                     deletes the file that it had saved)
Messages (3) and (4) do not actually lead to significant action in the save-to-Filer case, but in fact the same code also provides the save operation to another application, using a temporary file. A fuller discussion of this appears below, when the possibility of in-memory data transfer has also been introduced.

5 DataOpen (broadcast for double-clicked file)

This protocol is used to broadcast to all running applications just before "opening" a file whose icon has been double-clicked. It allows already-running applications to open the file instead. Typically this will be used by an editor which is capable of editing several documents of the same type, so that only one instance of the application runs (using much less space than if a separate copy of the application were run for each document).

   R0     18 (usually)
R1!0 size
     !12  0
     !16  5                 ; destination info request
     !20  window handle of dirviewer
     !24  0 ; icon handle not used
     !28  x-offset of icon being opened within viewer
     !32  y                 ; allows for 'zoom' box if implemented
     !36  must be <= 0 ; size field in other messages
     !40  file type
     !44  full path-name of file, 0-terminated.

DataLoadAck is returned by the application which loads the file. If this is not received, then the Filer will *Run the file.

An application directory that was double-clicked with the shift key held down, is broadcast as a directory.

If data is to be transferred between two applications without going out to a file, then the following reply initiates the in-core transfer protocol:

6 RAMFetch (transfer data to buffer in my workspace)

   R0     18 (error message if not acknowledged)
R1!0 size
     !12  my_ref field of DataSave message
     !16  6 ; RAM fetch.
     !20  buffer address
     !24  buffer size (bytes)

The original sender replies as follows:

7 RAMTransmit (I have transferred some data to a buffer in your workspace)

   R0     18 (error message if not acknowledged)
R1!0 size
     !12  my_ref field of RAMFetch message
     !16  7 ; RAM transmit.
     !20  buffer address             ; copy of value sent in RAM fetch
     !24  number of bytes written to buffer
          (if buffer not full, send another RAMFetch)

To write the data into the receiver's buffer, use the following call:

Wimp_TransferBlock
Entry: R0 = task handle of source
R1 --> source buffer
R2 = task handle of destination
R3 --> destination buffer
R4 = buffer length
buffer addresses and length are byte-aligned (not nec. word-aligned)
if the buffer addresses are within application space,
they are validated to ensure they are within the correct task
Errors: "Invalid task handle"

        "Wimp transfer out of range"

The receiver's buffer is not entirely filled then the receiver will assume that this is the end of the operation. No further confirmation is required.

If the receiver's buffer is filled then it will send a further RAMFetch message. This need not specify the same buffer or buffer size. Thus the operation continues until all data is transferred.

If the other end of the protocol does not answer then cancel the operation quietly, without generating an error message. The other end (e.g. if it ran out of space) will have already have complained.

Application code for direct file transfer

To save a file:

  1. Transmit DataSave message.
  2. If DataSaveAck returned, save the file. If there are errors in this process, report them to the user and cease the transaction. If the save succeeds, send a DataLoad.
  3. If DataLoadAck is not returned, (because the receiver is dead or badly written) then the sender should delete the file and report an error message saying "data transfer failed". Thus the Filer must acknowledge DataLoad, or else all files saved to it will be subsequently deleted.
  4. If RAMFetch returned, send RAMTransmit and loop until done. NOTE: all messages in this protocol apart from the DataSave should quote the
    other side's my_ref field in their your_ref fields, to ensure that the
           messages are acknowledged correctly.
    
    NOTE: In all cases where an unknown message action is received by an
           application, it MUST ignore the message completely.
    

    To receieve a file from another application:

    1. Receive DataSave message.
    2. If data can be loaded from RAM, send back a RAMFetch (look at approx data size in DataSave message, but do not rely on its absolute accuracy, ie. be prepared for MORE data than that to be sent). If RAMFetch not acknowledged, load from a file (step 3.)

      If RAMTransmit received,
      finished if buffer NOT filled,
      else send another RAMFetch and loop.
      If any RAMFetch other than the 1st is not acknowledged, report error "data transfer failed". (There's no need to check your_ref field if RAMFetch (18) used, since the RAMFetch will not be acknowledged.)

      3. If data must be in a file, return DataSaveAck "<Wimp$Scrap>". If DataLoad is received, load the file, delete it and return DataLoadAck. Note that the your_ref field of the DataLoad tells the receiver whether the file is the scrap file.

      Minimal functionality is for the sender to only cope with DataSaveAck, and for the receiver to only cope with file-based I/O. If the receiver wishes to engage in RAMFetch operations, it should be prepared for the sender to be ignorant of that protocol - ie. it should be prepared to revert to the scrap file mechanism.

      An Explanation of the Data Transfer Protocols

      Ignoring direct RAM transfer for the moment, all data transfer operations in the RISC OS desktop world can be accomplished using 4 message types. These are:

              DataSave
              DataSaveAck
              DataLoad
              DataLoadAck
      

      The various operations that an application must deal with are as follows:

              1a) Saving data to a file
              1b) Saving data to another application
              2a) Loading data from a file
              2b) Loading data from another application
      

      The protocols involved in each of these cases is described below - note that it is assumed that all replies carry the my_ref of the message they are replying to in their your_ref field:

      1a) Saving data to a file

      Application receives User_Drag_Box event (the file box has been dropped) Application calls Wimp_GetPointerInfo to find out where the pointer is Application sends Message_DataSave to the destination, with the leafname Filer replies with Message_DataSaveAck with "directoryname.leafname" Application saves data to the file
      Application sends Message_DataLoad to the Filer Filer replies with Message_DataLoadAck
      Everyone is happy.

      1b) Saving data to another application

      Application receives User_Drag_Box event (the file box has been dropped) Application calls Wimp_GetPointerInfo to find out where the pointer is Application sends Message_DataSave to the destination, with the leaf-name ExternalTask replies with Message_DataSaveAck with "<Wimp$Scrap>" Application saves data to the file
      Application sends Message_DataLoad to the ExternalTask ExternalTask loads and deletes the scrap file ExternalTask replies with Message_DataLoadAck Everyone is happy.

      2a) Loading data from a file

      Filer sends Message_DataLoad to the application Application loads the file
      Application replies with Message_DataLoadAck Everyone is happy.

      2b) Loading data from another application

      ExternalTask sends Message_DataSave to the application Application replies with Message_DataSaveAck to "<Wimp$Scrap>" ExternalTask saves data to the file
      ExternalTask sends Message_DataLoad to the application Application loads and deletes the scrap file Application replies with Message_DataLoadAck Everyone is happy.

      Summary

      As one would expect, (1b) and (2b) are symmetrical, so that any task can send data to any other, rather than some tasks being senders and some receivers. An additional advantage of this method is the overlap between file transfer between applications and file transfer to and from the Filer.

      Note that in protocols (1b) and (2b) it is the loader of the data who is responsible for deleting the scrap file afterwards, which is logical since it is the loader who invented the scrap file in the first place!

      The following summarises the application code necessary for loading and saving files:

      Loading files

      Received Message_DataSave: respond with Message_DataSaveAck to "<Wimp$Scrap>"

                                 make a note of the my_ref field of your reply
      

      Received Message_DataLoad: load the indicated file

                                 if your_ref = scrap my_ref, delete the file
                                 reply with Message_DataLoadAck
      

      Saving files

      Determine destination using drag operation followed by Wimp_GetPointerInfo Send Message_DataSave including your proposed leafname

      Received Message_DataSaveAck: save data to the indicated file

                                    reply with Message_DataLoad for same filename
      

      Received Message_DataLoadAck: usually ignored, but can be useful

      Direct RAM transfer

      For the experts, it is possible to indulge in more efficient data transfer where two applications are involved (ie. protocols (1b) and (2b)).

      Basically, instead of replying with Message_DataSaveAck to "<Wimp$Scrap>", an application should reply first with Message_RAMFetch, and if that is not acknowledged, revert to returning the conventional <Wimp$Scrap> message.

      Message_RAMFetch

              R0 = 18 (so that message must be acknowledged)
              R1!0   size
                !12  my_ref field of DataSave message
                !16  6 ; RAM fetch.
                !20  buffer address
                !24  buffer size (bytes)
      

      From the sender's end, it is permissible to totally ignore Message_RAMFetch, in which case the more usual scrap file method is used. If the RAM transfer protocol is understood, however, Wimp_TransferBlock can used to transfer data into the other task's workspace.

      After transferring the data, a RAMTransmit message should be sent. If the buffer was filled by the sender, the receiver should process the data and then send another RAMFetch. If the buffer is not full, this implies that the file transfer is complete, and there is no need for any further communication. Note that this means that if the file length fits into a whole number of buffers, then the last block must contain 0 bytes of data.

      Note that it is possible for an application to support RAM transfer on loading or saving or neither or both as required - indeed, it may be that an application can cope with direct RAM transfer for certain filetypes but not for others, and its capability may be different when loading and saving data. The protocols described allow for 'cop-out' by either the sender or the receiver of the data.

      New Summary

      Including the RAM transfer protocols, the application code looks like this:

      Loading files

      Received Message_DataSave: respond with RAMFetch to appropriate buffer

                                 if not acknowledged,
                                   respond with Message_DataSaveAck to "<Wimp$Scrap>"
                                   make a note of the my_ref field of your reply
      

      Received Message_RAMTransmit: process data in buffer

                                    if buffer was filled, send another RAMFetch
                                    otherwise all data has been sent
      

      Received Message_DataLoad: load the indicated file

                                 if your_ref = scrap my_ref, delete the file
                                 reply with Message_DataLoadAck
      

      Saving files

      Determine destination using drag operation followed by Wimp_GetPointerInfo Send Message_DataSave including your proposed leafname.

      Received Message_RAMFetch: call Wimp_TransferBlock to transfer data

                                 send Message_RAMTransmit for appropriate amount
                                 if buffer filled, expect a RAMFetch in reply
      

      Received Message_DataSaveAck: save data to the indicated file

                                    reply with Message_DataLoad for same filename
      

      Received Message_DataLoadAck: usually ignored, but can be useful