A CAMAC Data Acquisition system with CC7700 and PC-Linux

Naohito Iwasa

Cyclotron Laboratory, Institute of Physical and Chemical Research (RIKEN)

24 Dec. 1997

  1. Introduction
    Recently performance of personal computers have almost same quality as workstations have, and reliable multi-task operating systems (for example, Free-BSD, Linux) are available. We have produced a data-acquisition system with CAMAC and PC on the Linux operating system. (I tested on Slackware-3.4 + kernel 2.0.29-34.) The system cost about 700,000 Japanese-yen only (350,000 and 150,000 Japanese-yen for a CC7700 crate controller and its PCI interface board from the Toyo Technical co., respectively, 200,000 Japanese-yen for a PC, and 2,000 Japanese-yen for CDROM of SLACKWARE-3.4), and the performance is almost same as data acquisition systems with workstations. It takes about 7 micro second for a standard CAMAC cycle which is comparable to 2.5 micro second for start-burst auxiliary crate-controller [1] and 1.6 micro second for K3976 auxiliary crate-controller [2].
    The program is freeware. You may use it, and modify it on your own risk. When you need a help, I will help you as possible as I can. Your status report, suggestions and contributions on the program are welcome.

  2. Concept for the data acquisition system
    hardware
    Detail documents for CC7700 crate controller and its PCI interface board can be available from the Toyo Technical co. Since a interface board can drive up to 8 CC7700 crate controllers, we can easily build multi-crate system.

    software
    The data acquisition system was designed for that several analysis processes (analyzer) can be executed at same time. Undesirable destroy of control process or X-window system does not affect on the system.

    Linux is not complete real-time operating system. Therefore, in this system, acquisition process is included inside of the operating system as an device driver for speed up. A look-at-me signal from CAMAC modules makes activates interrupt routine (dc_int.c) via interrupt request signal (IRQ) on the PCI interface board. In the interrupt routine, you should describe what kind of data you want to measure and structure of event buffer (short int buffer[]). The event data are copied the 64 kwords buffer inside of the device driver which is read by a blocking process (dma_control). The blocking process gathers events to 16 kbyte blocks in the shared memory as well as stores a file in hard disk with the blocks when you need. The blocks in the shared memory can be read from other processes, for example, analyzing, status-report processes. The device driver and the blocking process are controlled by a control process (COM) via I/O-control functions to the device driver.

  3. How to install the program
    1. install slack-ware Linux to your PC.
      I checked it in Slackware-3.4, but don't in RedHat, Devian Linux.
    2. click here to get the source files of the acquisition system.
    3. unpack the package by the gunzip, and tar commands.
      (gunzip cc7700-1.1.1.tar.gz; tar xvf cc7700-1.1.1.tar or tar zxvf cc7700-1.1.1.tar.gz)
      the all the source files are copied on the cc7700/ directory.
    4. move to the cc7700/ directory
    5. change to super user (root) and type the command make devices
      This command makes a device for cc7700 of PCI (/dev/PCIcc7700).
    6. I recommend to add super-user bit for insmod, lsmod, and rmmod command by using
      chmod +s /sbin/insmod; chmod +s /sbin/lsmod; chmod +s /sbin/rmmod
      It might make a security problem, but if you did the procedure, you don't need super-user privillage to install or remove acquisition processes. (I wrote the document afterward assuming you did the procedure.)

    7. Preparation for your experiments
      1. move to the cc7700/ directory
      2. modify dc_int.c
        There are five subroutines in the file. Since these command were called in the device driver, you cannot describe the file input or output operation and floating calculation. Moreover, the variables should be defined with the "static" keyword.
        The dc_int_init() subroutine is called when you want to clear the status of the data acquisition system.
        The dc_int_start() and dc_int_stop() subroutine is called when acquisition process is started and stopped, respectively.
        The dc_int_clear() is automatically called just before the LAM interrupt process is finished for clearing the LAM signal as well as used modules.
        The dc_int(unsigned short buffer[2048]) subroutine is called when LAM interrupt happens.
        The buffer is event buffer (2 kword maximum) which will be transfer to blocking process. The word size (16 bit) should be written in buffer[0]. The other format are free. You consider it in your risk.

        There are several functions to communicate with CAMAC interface.
        void camac_CNAF(int C,int N,int A,int F)
        send CAMAC command with N, A, and F to the crate controller C.
        The interface can drive 8 CAMAC crate controllers.
        N:1-23, F:0-31, A:0-15, C:0-7
        void camac_write16(short data)
        write 16 bit data "data" to the CAMAC data-way for next CAMAC command.
        void camac_write24(long data)
        write 24 bit data "data" to the CAMAC data-way for next CAMAC command.
        short camac_read16()
        read 16 but data from the CAMAC data-way stored by previous CAMAC command and return the value.
        long camac_read24()
        read 24 but data from the CAMAC data-way stored by previous CAMAC command and return the value.

        Here is an example.

        #include "camack.c"
        
        #define INPUT_REGISTER  1       /* station number of input register   */
        #define INPUT_REG_RF    0       /* read function of input register    */ 
        #define INPUT_REG_RA    0       /* read sub-address of input register  */
        #define INPUT_REG_CF    9       /* clear function of input register   */
        #define INPUT_REG_CA    0       /* clear sub-address of input register */ 
        
        #define ADC             2       /* station number of ADC              */
        #define ADC_RF          0       /* read function of ADC               */ 
        #define ADC_CF          9       /* clear function of ADC              */
        #define ADC_CA          0       /* clear sub-address of ADC            */  
          
        #define SCALER          4       /* station number of scaler           */
        #define SCALER_RF       0       /* read function for scaler           */ 
        #define SCALER_CF       9       /* clear function for scaler          */
        #define SCALER_CA       0       /* clear sub-address for scaler        */
        
        #define OUTPUT_REGISTER 5       /* station number of output register  */
        #define OUTPUT_REG_F    17      /* write function of output register  */
        #define OUTPUT_REG_A    0       /* write sub-address of output register*/
        
        #define LAM_N           ADC     /* LAM source is ADC         */
        #define LAM_F           26      /* function for enable-LAM    */ 
        #define LAM_A           0       /* sub-address for enable-LAM */
        
        static long loop=0,data;
        static int stat;
        
        static void dc_int_init(),dc_int_start(),dc_int_stop(),dc_int_clear();
        
        static void dc_int_init()
        {
          camac_CNAF(0,SCALER,SCALER_CA,SCALER_CF);    /* clear scaler */ 
        }
        
        static void dc_int_start()
        {
          camac_write24(0x0000f0);			
          camac_CNAF(0,OUTPUT_REGISTER,OUTPUT_REG_A,OUTPUT_REG_F);
        				/* send signal to output register A=5-8 */
          dc_int_clear();			/* clear LAM and modules */
          camac_CNAF(0,LAM_N,LAM_A,LAM_F);  	/* enable LAM            */
        }
        
        static void dc_int_stop()
        {
          camac_CNAF(0,LAM_N,LAM_A,24);	/* disable LAM */
          camac_write24(0x000f00);
          camac_CNAF(0,OUTPUT_REGISTER,OUTPUT_REG_A,OUTPUT_REG_F);
        				/* send signal to output register A=9-12 */
        }
        
        static void dc_int_clear()
        {
          camac_CNAF(0,ADC,ADC_CA,ADC_CF);	/* clear data of ADC */
          camac_CNAF(0,ADC,ADC_CA,10);		/* clear LAM  of ADC */
          camac_CNAF(0,INPUT_REGISTER,INPUT_REG_CA,INPUT_REG_CF);
        					/* clear input register */
          camac_write24(0x00000f);
          camac_CNAF(0,OUTPUT_REGISTER,OUTPUT_REG_A,OUTPUT_REG_F);
        				/* send signal to output register A=1-4 */
        }
        
        static int dc_int(unsigned short buffer[])
        {
          long j; 
        
          loop++;
          j=inl(BASE_ADDRESS+4);  /* check LAM */
          if(j==0){
            printk("dc_int: interrupt %d happened, %d, LAM=%lx\n",IRQ,loop,j);
        			/* write warning for illegal LAM to dmesg */
            return(-1);		/* return                             */
          }
          buffer[0]=8;		/* fixed event length (8 words) */
          camac_CNAF(0,INPUT_REGISTER,INPUT_REG_RA,INPUT_REG_RF);
          buffer[1]=camac_read16();   /* first data is bit pattern of input register*/
          camac_CNAF(0,ADC,0,ADC_RF);
          buffer[2]=camac_read16();   		/* second data is ADC A=0 */
          camac_CNAF(0,ADC,1,ADC_RF);
          buffer[3]=camac_read16();   		/* the third data is ADC A=1 */
          camac_CNAF(0,ADC,2,ADC_RF);
          buffer[4]=camac_read16();   		/* the fourth data is ADC A=2 */
          camac_CNAF(0,ADC,3,ADC_RF);
          buffer[5]=camac_read16();   		/* the fifth data is ADC A=3  */
          camac_CNAF(0,SCALER,0,SCALER_RF);
          j=camac_read24();	      
          buffer[6]=j&0xffff;
          buffer[7]=(j>>16)&0xffff;   		/* 24 bit write for scaler    */
          return(0);
        }
        
      3. modify event.c
        There is a subroutine in the file. The subroutine is called from on-line analyzer process. Since it is not device driver, you can use any subroutines as you like. We recommend to use PAW which was distributed by CERN. Don't forget include package which include CFORTRAN. If you prefer to write on-line analyzer in FORTRAN, please modify event.f and Makefile.
        Unfortunately NTUPLE(CWN) format is not supported for GLOBAL section.

        Here is an example

        #include "/cern/97a/include/cfortran/cfortran.h"
        #include "/cern/97a/include/cfortran/hbook.h"
        		/* these include files are need for calling HBOOK subroutines
        		   from the C language */
        
        #define HBOOK_SIZE 200000
        
        void analyze_(unsigned short int data[])
        {
          static int loop=0;
          int i,j;
        
          if(loop==0){
            HLIMAP(HBOOK_SIZE,"SAMPLE");	     /* make global section "SAMPLE"*/
            HBOOK1(11,"SSD1R-L",4095,0.5,4095.5,0.0);/* define 1-dim histograms */
            HBOOK1(12,"SSD2R-L",4095,0.5,4095.5,0.0);/* see HBOOK manuals in CERN-lib*/
            HBOOK1(13,"SSD3R-L",4095,0.5,4095.5,0.0);
            HBOOK1(14,"SSD4R-L",4095,0.5,4095.5,0.0);
            HBOOK1(15,"SSD5R-L",4095,0.5,4095.5,0.0);
            HBOOK1(16,"SSD6R-L",4095,0.5,4095.5,0.0);
            HBOOK1(17,"SSD7R-L",4095,0.5,4095.5,0.0);
            HBOOK1(18,"SSD8R-L",4095,0.5,4095.5,0.0);
            HBOOK1(19,"SSDR-L",4095,0.5,4095.5,0.0);
            HBOOK1(21,"SSD1L-L",4095,0.5,4095.5,0.0);
            HBOOK1(22,"SSD2L-L",4095,0.5,4095.5,0.0);
            HBOOK1(23,"SSD3L-L",4095,0.5,4095.5,0.0);
            HBOOK1(24,"SSD4L-L",4095,0.5,4095.5,0.0);
            HBOOK1(25,"SSD5L-L",4095,0.5,4095.5,0.0);
            HBOOK1(26,"SSD6L-L",4095,0.5,4095.5,0.0);
            HBOOK1(27,"SSD7L-L",4095,0.5,4095.5,0.0);
            HBOOK1(28,"SSD8L-L",4095,0.5,4095.5,0.0);
            HBOOK1(29,"SSDL-L",4095,0.5,4095.5,0.0);
            HBOOK1(31,"SSD1R-H",4095,0.5,4095.5,0.0);
            HBOOK1(32,"SSD2R-H",4095,0.5,4095.5,0.0);
            HBOOK1(33,"SSD3R-H",4095,0.5,4095.5,0.0);
            HBOOK1(34,"SSD4R-H",4095,0.5,4095.5,0.0);
            HBOOK1(35,"SSD5R-H",4095,0.5,4095.5,0.0);
            HBOOK1(36,"SSD6R-H",4095,0.5,4095.5,0.0);
            HBOOK1(37,"SSD7R-H",4095,0.5,4095.5,0.0);
            HBOOK1(38,"SSD8R-H",4095,0.5,4095.5,0.0);
            HBOOK1(39,"SSDR-H",4095,0.5,4095.5,0.0);
            HBOOK1(41,"SSD1L-H",4095,0.5,4095.5,0.0);
            HBOOK1(42,"SSD2L-H",4095,0.5,4095.5,0.0);
            HBOOK1(43,"SSD3L-H",4095,0.5,4095.5,0.0);
            HBOOK1(44,"SSD4L-H",4095,0.5,4095.5,0.0);
            HBOOK1(45,"SSD5L-H",4095,0.5,4095.5,0.0);
            HBOOK1(46,"SSD6L-H",4095,0.5,4095.5,0.0);
            HBOOK1(47,"SSD7L-H",4095,0.5,4095.5,0.0);
            HBOOK1(48,"SSD8L-H",4095,0.5,4095.5,0.0);
            HBOOK1(49,"SSDL-H",4095,0.5,4095.5,0.0);
          }
          loop++;
          
          i=data[1]&0x07;			/* the event data are stored in data */
          HF1(11+i,(float)data[2],1.0);		/* fill histograms */
          HF1(19,(float)data[2],1.0);
          HF1(31+i,(float)data[3],1.0);
          HF1(39,(float)data[3],1.0);
          i=(data[1]>>3)&0x07;
          HF1(21+i,(float)data[4],1.0);
          HF1(29,(float)data[4],1.0);
          HF1(41+i,(float)data[5],1.0);
          HF1(49,(float)data[5],1.0);
        }
        
      4. type make
        The command makes executable files from source files. Two warnings will be appeared in PCIcc7700.c, but you don't mind the message.

    8. start acquisition system
      1. move to the cc7700/ directory
      2. type start_daq
        The command installs device driver and makes child processes. Maybe two windows appears in your display. One (COM) is for controlling the system. The other (STA) is for monitoring the system. You can quit the program without affecting the system by typing Cntr-D or Cntr-C, respectively. When you need the window, type com or sta in the cc7700 directory.
        If the message ``Initialization of PCIcc7700 failed'' appeared or hang it up, please check ``/proc/pci'', ``/proc/ioports'', and ``/proc/interrups'' files as the conflict I/O ports, interrupt-request number, and so on. Please solve the problem by modifying Makefile. If you cannot solve to problem, I recommend you to change the I/O port address because the I/O port used in mother board or Bios does not appear the list for ``/proc/ioports''.

    9. How to control acquisition system (COM)
      1. activate COM process.
        In case of finding no window for ``COM'' process, please make a xterm (or kterm) window in your display, activate the window, move to the cc7700/ directory, and type com.
      2. when com asks as ``Scaler??: station number (end=0)'', type station number for scalers. When you finish to describe the scalers, type carriage return. (check STA window!)
      3. Probably ``COM'' prompt appears.
      4. when you want to modify scaler station numbers, type set.
      5. when you want to start the acquisition process, type start.
        (In the STA window, the start time and message of ``acquiring'' are written)
      6. when you want to stop the acquisition process, type stop.
        (In the STA window, the stop time is written)
      7. when you want to clear the status, type clear
      8. when you want to write raw data to disk, type open
        Then filename and comments are required. (In the STA window, filename appears) Then type start
      9. when you want to finish to write raw data to disk, type stop
        Then type close. Comments are required. (In the STA window, filename disappears)
      10. when type sca, data of all scalers are printed on the window.
      11. when type carriage return, event numbers and buffer numbers are printed.

    10. How to see histograms on line.
      1. make a new xterm (or kterm) window.
      2. type ``paw''
      3. PAW asked you to type ``workstation type''. If your display supports X-window protocol, type 1.name-of-display
      4. Probably, PAW prompt appears
      5. type global_sec name
        the name should be same name describe in ``event.c''
      6. type h/list, then you will see numbers and titles for all histograms.
      7. type h/pl number, then histogram will appears in HIGZ window.
      8. when you want to erase histograms, type sh erase number[ number] for the paw prompt or erase number[ number] for shell prompt
        number=0 means to erase all histograms.
        Note that greset command of PAW cannot be used in some version of Linux kernel.

    11. Trouble shooting
      • in the case of hanging-up of the com process
        Please type ``ps -x|grep dma_control|grep -v grep''. When there is no process for dma_control, type restart_dma in the cc7700/ directory. In the case of hanging-up of window, please check you can log-in from the other terminals. When yes, please kill the X-window process and restart it. If not, please reduce event rate. If the problem is not solved unfortunately, please reboot the computer. Of course, before re-starting the data acquisition system, please check dc_int.c file as not to use large local variables.
      • When on-line analyzer doesn't work, although data was acquired
        Please type ``ps -x|grep analyzer|grep -v grep''. When there is no process for analyzer, type restart_ana. If not, please start another paw session to check if the same situation appears. In this case, please check consistence of event structure between event.c and dc_int.c.

Acknowledgment
I would like to thank Institute of Physical and Chemical Research (RIKEN), especially Prof. Yano and Dr. Morita (Cyclotron Laboratory, RIKEN) for their support for producing the data acquisition system. I am grateful to Dr. Okamura (Univ. of Tokyo) and Dr. Yasu (KEK) for their providing source programs of their data acquisition systems and for their suggestions. The device driver was developed from device driver for ISAcc7000 made by Dr. Okamura. I thank Drs. Ichihara, Watanabe, and Nakamura[2] for their providing information on the existing data acquisition system.

APPENDIX
addressBASE ADDRESS+0BASE ADDRESS+4BASE ADDRESS+8
BITwritereadwritereadwriteread
31------
30------
29------
28------
27------
26------
25------
24------
23---LAM station 24 data bit 24data bit 24
22---LAM station 23 data bit 23data bit 23
21---LAM station 22 data bit 22data bit 22
20---LAM station 21 data bit 21data bit 21
19---LAM station 20 data bit 20data bit 20
18--crate Address 04 LAM station 19data bit 19data bit 19
17--crate Address 02 LAM station 18data bit 18data bit 18
16--crate Address 01 LAM station 17data bit 17data bit 17
15--- LAM station 16data bit 16data bit 16
14--- LAM station 15data bit 15data bit 15
13--station number 16 LAM station 14data bit 14data bit 14
12--station number 08 LAM station 13data bit 13data bit 13
11--station number 04 LAM station 12data bit 12data bit 12
10--station number 02 LAM station 11data bit 11data bit 11
9--station number 01 LAM station 10data bit 10data bit 10
8--subaddress 08 LAM station 9 data bit 09data bit 09
7LAM internal LAM internal subaddress 04 LAM station 8data bit 08data bit 08
6reset LAM sum subaddress 02 LAM station 7data bit 07data bit 07
5- online subaddress 01 LAM station 6data bit 06data bit 06
4- done function 16 LAM station 5 data bit 05data bit 05
3enable interruptenable interruptfunction 08 LAM station 4 data bit 04data bit 04
2inhibit inhibit function 04 LAM station 3 data bit 03data bit 03
1initialize no-Xfunction 02 LAM station 2 data bit 02data bit 02
0clear no-Qfunction 01 LAM station 1 data bit 01data bit 01
From the manual of the cc7700 crate controller

References
[1] T. Ichihara, T. Inamura, T. Wada, M. Ishihara, IEEE Trans. Nucl. Sci. 36 (1989) 1628.
[2] S.N. Nakamura and M. Iwasaki, Nucl. Instr. Meth. A388 (1997) 220.