CSIM as a Performance Modeling Tool:
An Overview

Table of Contents

1. Choosing CSIM for Performance Modeling
2. CSIM Performance Model Distribution: Setup & Primer
   • 2.1 Installation
   • 2.2 Building Hardware and Software Models
   • 2.3 Running Simulations
   • 2.4 Analyzing Results
3. An Overview of the CSIM Tools
   • 3.1 CSIM PreProcessor
   • 3.2 Behavioral Description
   • 3.4 Topological Description
   • 3.5 CSIM GUI
   • 3.6 Scheduler
   • 3.7 Router
   • 3.8 Analysis Tools
   • 3.9 Device Models
4. A Performance Model Demonstration
   • 4.1 A Simple Architecture File: arch1.sim
   • 4.2 A More Complicated Architecture File: arch2.sim
   • 4.3 A Simple Data Flow Graph: flow1.dfg
   • 4.4 A More Complicated Data Flow Graph: flow2.dfg
   • 4.5 A Node to Device Mapping File: flow2map.csim
   • 4.6 Running the Demonstration Performance Models
5. Appendix
   • A. CSIM Distribution Directory Tree
   • B. CSIM Device-Level Behavioral Description
   • C. Software Commands Processed by the Programmable Class
   • D. Change Record

1. Choosing CSIM for Performance Modeling

Why Choose CSIM?

Lockheed Martin Tactical Systems, Naval Electronics & Surveillance Systems (Eagan, MN) (LMTS) has considered several modeling languages for these sorts of tasks including Extend, Foresight, Opnet and ATL's CSIM. (BONeS was not considered because it is no longer supported.) CSIM meets all of the above requirements and, with additional modifications made by Lockheed Martin's Advanced Technology Laboratories (ATL), has met all of our needs. First of all, CSIM's design easily allows for a separate construction of hardware and software models. Software models can be 'hosted' on a changing hardware model through use of a mapping file that associates software processes with hardware devices (details below). This means that software can easily be reallocated to different hardware. It also means that software designers at one location can 'test' (coarse-level) software performance on earlier hardware models even as those hardware models are being modified at another location. Likewise, hardware modelers can test new hardware models through use of existing, but not necessarily up-to-date software models.

Secondly, and in contrast with other considered modeling languages, all CSIM models can be constructed more naturally from a top-down rather than bottom-up approach. Hardware models can be specified at a system level first, then for each system module, and finally at the level of each device within the modules. Similarly, software models can be specified in terms of a computer software configuration item (CSCI) followed by constituent software components (CSC) and finally in terms of software units (CSU). The top down approach mimics the way large systems are typically designed and primarily means that the bottom-level details can be changed as new information becomes available without major revisions of the model.

Thirdly, should the need arise (i.e., in the event ATL fails to support CSIM), Lockheed Martin programs will gain control of CSIM's current source code. Such a failure of support situation has already happened when the company who owned "BONeS" discontinued its support. In the CSIM case, not only is Lockheed Martin the owner, but the CSIM source code is designed in such a way that one or two individuals can maintain it.

In general, one of CSIM's greatest strengths is in its role as a Systems Engineering tool. The process of constructing hardware and software models requires a level of communication between the various software and hardware teams. The model itself represents a way to accumulate and disseminate hardware and software structure and interaction information. Design and performance weaknesses can be exposed at the earliest stages of development allowing for corrections that have the lowest impact on cost and schedule.

Why Not Choose an Alternative?

As mentioned, several alternative modeling languages were considered for performance modeling. Opnet and Foresight (and BONeS) are more suited for lower-level modeling tasks, such as bit-level, functional and detailed interface modeling. This is not so much because of their languages, but because many time-consuming validated models exist in these modeling languages. In fact, ATL uses some of these other modeling tools to validate their models.

We are concerned with situations where both architecture and software designs may change in significant ways before the program ends. Without design and device details, lower-level models tend to provide pointlessly overly precise information about incorrect models. These alternative modeling languages also strongly mix software with hardware design. This creates configuration management nightmares when teams across the country have to incorporate changes to their modeled components simultaneously. Software and hardware coupling also makes it difficult to swap in and out alternative scenarios; modeling of a hardware failure or modeling situations including or excluding various software functions. The alternative tools do allow for the development of software schedulers and message routers, but the construction of these tools would be awkward and development very time consuming.

The third modeling language considered closely by LMTS, Extend, is better suited for simpler systems. Conversely, it is inefficient for more complex systems. By way of example, one specific avionics model developed by LMTS and the entire CSIM modeling language can fit onto a 1.4MB floppy disk and be run on just about any standard computer. A one-tenth second CSIM simulation of this model takes approximately three minutes. A part of this system, the multi-priority PE and software frame-rate model (discussed below) was created in Extend by another company. In addition to product installation that requires a CD, Extend requires a 100MB Zip disk to hold the compressed models, and a 250MB RAM 200+MHz Pentium to run. Typical run times consume days, even on a much more powerful PC than above!

Modeling Goals

The primary goal considered here is to provide a performance modeling tool that helps our Systems Integration team verify that all system components are likely to work together. This includes looking at metrics such as signal contentions, message transfer delays, component utilization under various scenarios, and in particular identifying any system bottlenecks. Performance models also can be used to analyze system responses to component and certain high-level software failures, to track changes in system design, and test hypothetical alternative architectures and devices.

The standard CSIM distribution consists of a single file named csim_install_?.tar, where the ? is the version number. The contents of this self-contained, compressed archive file include:

• The most recent version of ATL's tool distribution supporting Sun Solaris, PC-Linux, SGI-Irix, HP-UX, and Mac-OSX.
• Several libraries of re-usable general purpose models. And,
• A set of simple CSIM demonstration models and examples.

2.1 Installing or Updating CSIM, and Setting Environment

Note:   Sites with a shared file system need install only one copy of CSIM which can be shared by all users. In other words, only one person at each site needs to perform this install; not each user. We recommend appointing this person as custodian of the installation. The custodian(s) can efficiently handle tool updates and configuring the individual user's environments. (Of course, there is no harm in installing multiple copies, it is just not as efficient to maintain.)

(1) Recommendation:   Be in a C-shell or compatible shell. The tcsh shell is preferred. If this isn't your default shell, simply type tcsh or csh. (Although any shell can be used with CSIM, the instructions here apply to tcsh or csh. These instructions have been tested in these shells.)

(2) Go to the directory where you wish to install CSIM.
Example:
        cd /proj/xx

(3) Type:
        tar xvf install_v*.tar
and,
        gunzip -r csim

This will unpack and uncompress the CSIM files into a directory called csim under your current directory. (i.e. /proj/xx/csim ) This forms the CSIM root location. The files in the csim directory will have a directory structure similar to the one in Appendix CSIM Distribution Directory Tree.

Now you have installed the CSIM files. Next, you need to adjust the setup according to the environment at your site.

(4) Edit the csim/tools/setup file. Look for the keyword "CSIM_ROOT" and change this directory path to reflect your installation location. Also look for your machine's "CSIM_C_COMPILER" variable, and, if necessary, change any "gcc" entries to the "cc" command of your local compiler.

(5) You may want to edit csim/tools/platform/gui_setups file, where 'platform' corresponds to the type of your system(s), if your local text editor is not the stated default ("textedit" or "nedit" respectively). A common alternative "xterm -e vi" will work on most systems. (This launches the vi editor within an xterm-window.)

(6) Source the csim/tools/setup file.
Example:         Source         /proj/xx/csim/tools/setup
"Source" is a csh/tcsh command that executes a script file. In this case, it tells your shell where the appropriate CSIM tool executables are for your platform, and makes quick aliases for them.

This final step (6) is the only one which must be repeated for each session and user of CSIM. For convenience, we recommend placing the source command in your home directory .cshrc or .tcshrc file, to occur automatically every login.

Install Note 1:   If you are using an SGI machine, you may need to follow this additional step. Some SGI machines have a built-in artificial posix-thread limit of 1,024. This is more than sufficient for all of the CSIM tools and demonstration models. However, if you build very large architecture models, you may reach this limit on SGI systems, which would be evidenced by your simulation freezing. The limit can be increased arbitrarily by your system administrator (refer to system man-pages for 'ulimit' and 'setrlimit'). If necessary, feel free to contact Carl Hein at ATL, or Keith Burgess at LMTS, for specific SGI thread modification instructions.

Install Note 2:   A similar issue sometimes applies with Linux, depending on the version installed. If the Linux was installed for i386 compatibility, there may be a limit of 2,048 threads, and the file /usr/include/bits/local_lim.h may have POSIX_THREAD_THREADS_MAX set to only 64 as a default. We recommend you increase this higher, such as 2,048. Likewise the quantity NR_TASKS in /usr/src/linux-2.2.5/include/linux/tasks.h may need to be increased from its default of 512 to 2,048 and the default stack size per thread process may need to be diminished via the setstackaddr attribute. These limits were found in older distributions of Linux. Newer versions, or installations for Pentium class processors, tend to have these limits set much higher by default. Contact Keith Burgess for specific Linux modifications, if necessary.

Install Note 3:   If you are using CSIM through Exceed on a PC, there is an Exceed bug noted in the CSIM FAQ, that sometimes prevents the display of tick-marks (object-selection/highlight marks) on selected objects in the GUI. It also prevents seeing the "rubber-band" stretch lines when dragging a selection box, drawing a link, etc. This can create much confusion if you don't know about it. The current remedy is to set your MS-Window's color-mode to 8-bits (256-colors). This remedies the problem. Also, remember that Exceed screen refresh limitations do slow down screen updates. This can be especially notable when the displaying detailed timelines with the XGraph tool.

2.2 Building Hardware and Software Models

The CSIM distribution includes a number of demonstrations in the directory csim/demo_examples. This section focuses on the performance modeling demonstration discussed further in More on the Performance Model Demonstration. This model includes two 'hardware' models and two 'software' models and resides in the demo directory 'csim/demo_examples/perfmodel'. Below are instructions for building the simpler of two software models on the simpler of two hardware descriptions.

Basic Setup

(1) Make sure you have sourced the csim/tools/setup file, as described in step (6) of the installation above.

(2) Create a directory where you have write access. Then copy the example performance model directory contents to that directory:
Example:
        mkdir   ~/myperfmodel
        cp   -r   $CSIM_ROOT/demo_examples/perfmodel   ~/myperfmodel

(3) Move into your directory where the models are:
Example:
        cd   ~/myperfmodel

(4) Open the GUI on the arch1.sim file.
Type:
        gui   arch1.sim
This will open the simpler of two 'hardware' architecture models in the CSIM graphical tool and display a diagram similar to that in Figure 1.

Figure 1 - arch1.sim.   A Simple Hardware Model.

(5) Choose the menu item "Tools->Build Simulation" to compile the HW architecture model.

(6) Choose the menu item "Tools->Build Routing Table" to construct the network routing information tables.

Alternative Build Note:   Steps four through six can be substituted for the following three non-graphical commands executed from the C-shell:
        csim   arch.sim
        sim.exe   -netinfo
        router   netinfo
Nothing about CSIM requires use of the GUI. Non-graphical commands are often quicker.

(7) Choose "File->Open->Open a new file" and choose the file "flow1.dfg". This provides a graphical view of the simpler of two SW models and yields a diagram similar to that in Figure 2.

Figure 2 - flow1.dfg.   A Simple DFG.

(6) Choose the menu item "Tools->Build DFG SW" to build the simple 'software' model.

(7) Choose the menu item "Tools->Plot Ideal TimeLine" to see the ideal timeline.

2.3 Running Simulations

(1) The performance modeling simulation can be viewed from the perspective of either the hardware or software. By default, the simulation is viewed from the hardware perspective and the default graphical window depicts the hardware architecture: a graphical description similar to Figure 1.

This can be changed to view the simulation from the data flow graph (DFG) representation (a view similar to that in Figure 2). To accomplish this, type setenv SIM_GRAPH flow1.dfg before starting up the GUI.

The simulation view can be changed back to the hardware perspective (before starting the GUI) by typing unsetenv SIM_GRAPH or equivalently setenv SIM_GRAPH arch1.sim.

(2) If not in the GUI, type gui arch1.sim or gui flow1.dfg. Choose the menu item Tools->Run Simulation. This step could equally have been executed from the command line via sim.exe.

(3) Optionally choose Animation->Animation Types->Nodes: Concurrent Activities and/or choose Animation->Animation Types->Links: User/Model Defined. These commands override default device and node coloring discussed in Run-time analysis below and are less useful for models (such as these) that use "core_models" components.

(4) Click on "Run/Continue".

(5) When asked (at the UNIX shell window), choose a verbosity level. This verbosity level controls the detail of command-line feedback the simulation will provide about the state of messages in the simulation. Zero is typically chosen to minimize output and speed up the simulation.

2.4 Analyzing Results

The simulation shows the flow of messages from creation to destination by coloring the various device and DFG objects. The simulation can be slowed down by adjusting the "Speed Slowdown" slider, stopping, stepping through the simulation, or crawling through the simulation per simulation control panel buttons. These and other GUI-accessible controls are described in ATL's documentation (see The CSIM Graphical Simulator).

When viewing a general simulation or plotted output, colors have the following meanings:

HW Raceway and Fibre Links are:

• Purple when unused. They are thin before their first use or when a window is refreshed. They become thick after any use and remain that way until the window is refreshed (usually by changing views).
• Yellow on each segment where a packet request has been made and where each wormhole has been established. The yellow back tracks leaving purple behind its retreat-all the way to its destination if the wormhole is denied.1
• Orange is shown for control messages (such as token transfers) in Fibre models.
• Blue when the first packet of a given message block passes across a link.
• Green if data flows down a segment unencumbered.
• Red to show contention if data is already flowing down a segment when a second data stream attempts to create a wormhole down that same segment.

HW Generic XBar Links are:

• Purple when unused.
• Green when data is flowing through the device. (There is no contention on this XBar model.)
• HW Raceway XBar, Fibre XBar and NIC/NIU devices are:
• Black (uncolored) when unused.
• Blue when the device is processing its first packet of a given message block.
• Green if handling a control message (not implemented consistently).
• Red when the device is being used by further packets.

HW Devices, HW Modules, SW Nodes, and SW Supernodes are:

• Light cyan with a thin border for devices and nodes and a thick border for modules and supernodes.
• Blue when they initiate the first packet of a given message block.

Other devices listed in the timeline plots including the generic_pe and multi_priority_pe are:

• Color-coded by a mapping from the LAST letter or digit of the data flow diagram node name that is providing the device's instructions. The mapping is made according to Table 1 (copied from subroutines.sim).
• Timeline plot devices supporting an unnamed monotonic (an alarm that goes off periodically to restart a data flow: say at 33.3ms) are pink.

Table 1 DFG Task Node Name Color Code Mapping

SIM-Display Panel colorize() value	Last Character of Task Name	XGRAPH Color (colormap() value)
--None--	8,F,T,b,p	0 Black
1 Fuchsia	8,F,T,b,p	12 Fuchsia
2 Blue	9,G,U,c,q	3 Blue
3 Cyan	H,V,d,r	9 Cyan
4 Navy	I,W,e,s	14 Navy
5 Yellow	J,X,f,t	7 Yellow
6 Dark-Gray	K,Y,g,u	11 Dark-Gray
7 Gray	O,L,Z,h,v	10 Light-Gray
8 Red	1,M,i,w	2 Red
9 Green	2,N,j,x	4 Green
10 Violet	3,A,O,k,y	5 Violet
11 Orange	4,B,P,l,z	6 Orange
12 Gold	5,C,Q,m	15 Gold
13 Pink	6,D,R,n	8 Pink
14 Dark Cyan	7,E,S,a,o	13 Aqua
15 White	-- None --	1 White

In addition to an object's color, a link's simulated values can be obtained during simulation runtime by choosing a link and then choosing "Options->Examine Link". Similarly, a list of all active links can be obtained by choosing "Options->List Active Links".

(1) To view the run-time-generated process timeline, choose "Tools->Plot Proc Timeline" from the GUI. Equivalently, type "xgraph ProcTline.dat &" from the command line.

(2) To view the run-time-generated timeline as well as the network utilization paths, choose "Tools->Plot Comm+Proc Tline" from the GUI. Equivalently, type "xgraph Spider.dat ProcTline.dat &" from the command line.

(3) To view system-wide contention levels, type "view_contention LinkTline.dat" from the command line. (This step can not easily be processed directly from the GUI.) The view_contention executable creates a file 'LinkTline.hst' that can be viewed graphically with the command "xgraph LinkTline.hst". Other contention analysis options are described at http://www.atl.lmco.com/proj/csim/view_contention.html.

(4) To create a specialized plot of simulation events, use the event tool. Like the contention analysis tool, this involves processing a text file: type "timeline EventHist.dat" and respond to a series of data-generated questions. The results are then viewed by entering "xgraph EventHist.tln". This is a powerful tool but it is also fairly complicated. Instructions for its use are available at www.atl.lmco.com/proj/csim/timeline/timeline.html.

(5) View the ASCII file 'summaries.dat' for processor, link, and port utilizations. Note that these statistics are collected between the default times "Time1" = 0 msec and "Time2" = 1,000,000 msec. This time window can be changed by editing the file "csim/model_libs/core_models/parameters.sim".

(6) To create a supplemental event history file, run the simulation at high verbosity and pipe the output to a separate file. This is accomplished as follows:

1. From the GUI, choose "Tools->Modify Commands->Run Simulation" and alter the command line to be "./sim.exe -V 10 > simoutput.txt &". Alternatively, type "sim.exe -V 10 > simoutput.txt &" directly from the command line. Choose a number more or less than ten to increase or diminish the quantity of information output.

2. View this (usually enormous) file directly or use the grep tool to filter the information. For instance, to obtain a file containing all activity on a device 'source', type "grep source simoutput.txt > anotherfilename.txt". To view all of the activity occurring to a message with ID number 2, type "grep mid=2 simoutput.txt | more".

A well-written manual can be seen on-line at www.atl.lmco.com/proj/csim. Frequently asked CSIM questions are available at www.atl.lmco.com/proj/csim/faq.html and www.atl.lmco.com/proj/csim/faq2.html.

ATL does have an "Issues Database" at www.atl.lmco.com/proj/csim/issues. A number of issues and resolutions have been posted here.

For general CSIM modeling questions call Carl Hein or Aron Rubin at ATL. For performance modeling CSIM modeling questions, feel free to call Keith Burgess or Ray Artz at LMTS.

Carl Hein (ATL) chein@atl.lmco.com
Aron Rubin (ATL) arubin@atl.lmco.com
Keith Burgess (LMTS) keith.burgess@lmco.com
Ray Artz (LMTS) ray.e.artz@lmco.com

image007.jpg

image009.jpg

image011.jpg

image013.jpg

Lockheed Martin Tactical Systems, Naval Electronics & Surveillance Systems; P. O. Box 64525; St. Paul, MN 55164-0525.
Keith Burgess: 651-456-2454, keith.burgess@lmco.com.
Ray Artz: 651-456-2328,