Chapter

6
WindView 2.0

Tornado Training Workshop

Copyright Wind River Systems Wind River Systems

6-1

Introduction

Conguring WindView

Events

WindView 2.0
6.1 Introduction Conguring WindView Events

Tornado Training Workshop

Copyright Wind River Systems Wind River Systems

6-2

What is WindView?
A software logic analyzer or run-time analyzer for your multitasking application. A high precision graphical view of events, tasks, and interrupts in your application, shown across time. Helps diagnose race conditions and timing problems. Data logging can be triggered to focus on events of interest. Other programs may access your event log using C++ or Tcl programmatic interfaces to the event base. WindView for VxSim Lite comes with Tornado. For other targets, WindView is an optional product.
Tornado Training Workshop Copyright Wind River Systems Wind River Systems

6-3

Host/Target Architecture
Host Target VxWorks and Application

File

Events WindView Graph

Event Buffer evtRecv tWVUpload tWvRBuffMgr

Tornado Training Workshop

Copyright Wind River Systems Wind River Systems

6-4

When event logging is enabled, event data are placed in a buffer in target memory as system instrumentation points and user events are encountered. This buffer may be dynamically extended or contracted by the task tWvRBuffMgr.

The task tWVUpload uploads event data to the host. When and how this upload occurs may be congured.

Events may be uploaded to a listening WindView graph, to the evtRecv tool, or to a host le, by various different upload methods.

Event logging, event upload, and event buffer management may be congured either programmatically or via the Tornado GUI.

Conguring WindView
Conguring WindView involves answering several questions:
q q

When does event logging start and stop? Which events are logged? How is the event buffer managed? When are event data uploaded to the host? How are event data uploaded to the host? What destination will receive the event data? Conguring VxWorks and the target server Using the WindView and Triggering host tools Calling target routines to control logging and upload
Copyright Wind River Systems Wind River Systems

Providing the answers to these questions may involve:

q q

Tornado Training Workshop

6-5

Target Server File System (TSFS)

Extends the virtual I/O facilities provided by the target server and WDB agent: Provides remote access to host lesystem. Target server can also open TCP socket connections on the host on behalf of target. Both methods are available for event log upload.
q q

The user must congure the target server to enable TSFS.

q q

Specify le system root directory. Specify read or read/write access. Write access is required for the TSFS event log upload to a le.

TSFS component of the WDB agent must be included.

Tornado Training Workshop Copyright Wind River Systems Wind River Systems

6-6

The TSFS facilities may be used even on targets without networking support (such as the bundled version of VxSim) so long as there is a communication channel between target server and WDB agent.

When the TSFS component of the WDB agent is included in VxWorks, an I/O system device named /tgtsvr is created on the target. This device allows access to TSFS facilities.
q

To open a le foo/bar on the host under the TSFS root directory, one opens /tgtsvr/foo/bar on the target. Operations on the returned le descriptor are actually handled by the target server, via the WDB agent. To create a TCP socket connection to port number 6164 on host do, one opens the le /tgtsvr/TCP:do:6164 on the target, obtaining a le descriptor which is for most purposes equivalent to a socket. The actual socket, however, is created by the target server on the host.

See the reference pages for wdbTsfsDrv for more information.

Your Application VS. tWVUpload

Uploading intrudes on the running of your application, as does buffer management. Use triggering to start WindView, focusing on the area of code you are interested in; use deferred upload mode so that the upload occurs after the relevant portion of the code is done and event data are gathered. Use post-mortem mode if you expect a crash and want to see what is happening just prior to the error. Stop dynamic buffer management by dening the minimum and maximum number of buffers to be the same.
Tornado Training Workshop Copyright Wind River Systems Wind River Systems

6-7

Buffer thrashing, a condition in which the ring buffer is repeatedly extended and contracted as the amount of data crosses a threshold, may sometimes occur during continuous upload mode. Often it can be avoided by increasing the minimum buffer count beyond the thrashing point.

wvUploadTaskCong (stackSize, priority) can be used to modify the priority of the upload task for your next run.

WindView 2.0
Introduction 6.2 Conguring WindView Events

Tornado Training Workshop

Copyright Wind River Systems Wind River Systems

6-8

Conguring VxWorks for WindView

Tornado Training Workshop

Copyright Wind River Systems Wind River Systems

6-9

The windview class instrumentation and windview library components will be required whenever using WindView.

One must select one timestamping component. The sequential timestamping component assigns only sequence numbers to events, so that time information beyond simple ordering is not recorded. The system-dened timestamping component requires support for high resolution timestamping within the BSP. An application note on writing timestamp drivers for custom BSPs is available.

One must select at least one upload path. The TCP/IP socket upload path requires network support on the target. The TSFS (socket) upload path requires support for the Target Server File System within the WDB agent. The le upload path requires a le system accessible to the target, often NFS or TSFS which allow writing les to a host le system.

Which Events are Logged?

There are three nested classes or levels of events which may be be logged: Context switch events Task state transition events Additional instrumented events Each class includes the preceeding classes.
q q q

At the Additional Instrumentation level, one may select which among several instrumented VxWorks Libraries will log events. Programmatically, one may also
q q

Instrument particular objects. Log user events with wvEvent().

Copyright Wind River Systems Wind River Systems

Tornado Training Workshop

6-10

Logging larger classes of events yields more information, but imposes greater overhead. The amount of overhead which WindView imposes on the target system is also affected by
q

The amount of instrumented (kernel) work in the application compared to non-instrumented work. The upload strategy. The event buffer conguration.

Most of the programmatic routines used for controlling instrumentation and logging are implemented in wvLib. See the VxWorks Reference Manual entry for this library for more information.

WindView Collection Conguration

1. Click Launch WindView toolbar button...

2. WindView Control panel and Collection Conguration dialogs appear...

3. Select event logging class.

Congure WV Button opens Collection Conguration Dialog after WindView is already launched.

For Additional Instrumentation logging class, select libraries for which to enable event logging.

Tornado Training Workshop

Copyright Wind River Systems Wind River Systems

6-11

When logging at the Additional Instrumentation level, selecting a library for logging means that all objects of that library will log events. For instance, selecting semLib means that all object events involving any semaphore in VxWorks or your application will be logged. To instrument only some objects of a given type requires that wvLib routines be called in your application (or from the shell). See the reference entries for wvObjInst() and wvObjInstModeSet() for more information on logging events corresponding to particular objects or sets of objects.

Upload Paths and Destinations

WindView denes several upload paths
Upload Path Direct to Graph Socket via TSFS File via TSFS File via NFS Destination Automatically opened WindView graph. Listening WindView graph or evtRecv. Host le writable via TSFS. Any le writable by target.

Socket via TCP/IP Listening WindView graph or evtRecv.

Except for the Direct to Graph path, one must specify additional information on the upload destination:
q q

host and port number for socket upload paths. directory and le name for le upload paths.
Copyright Wind River Systems Wind River Systems

Tornado Training Workshop

6-12

Direct to Graph upload is essentially equivalent to Socket Via TSFS upload; however, it automatically opens a listening WindView graph and assigns appropriate port numbers. This is the default upload path.

For upload via TSFS socket, one must specify the target server device path for the destination host and port number, e.g. /tgtsvr/TCP:myhost:6164.

For upload via TCP/IP socket, one must specify the destination host and port number, e.g. myhost:6164.

For upload to a le via TSFS, one must specify the TSFS path to the le on the host, e.g. /tgtsvr/evtLog.wvr; one must also congure the target server specifying the root of the TSFS le system, and giving read/write TSFS access.

Using File via NFS, one species any le writable by the target.

Upload Mode
Deferred upload:
q q

Data are uploaded on request after logging stopped Logging also stops if event buffer becomes full. Data are uploaded periodically as logging continues. Ring buffer grows during times of greater event production, shrinks once data are uploaded. Logging stops if event buffer becomes full. Data are uploaded on request after logging stops, often after warm reboot. Event buffer is treated as circular. Buffers must be in memory not overwritten during the reboot.
Copyright Wind River Systems Wind River Systems

Continuous upload:
q q

Post mortem upload:

q

Tornado Training Workshop

6-13

For deferred and continuous upload, logging stops if the buffer becomes full, that is, if either the maximum buffer count is reached, or if memory becomes exhausted to the granularity of the individual buffer size, or if new buffers cannot be allocated fast enough to store events.

In deferred and post-mortem mode, it is recommended that you stop logging (if it is going on) before clicking the upload button .

Conguring VxWorks for post-mortem mode requires specifying the location of the event buffer and ensuring that neither VxWorks nor the boot program overwites this memory when the target reboots. This is discussed in the WindView Users Guide. A common technique is to specify the macro USER_RESERVED_MEM to reserve memory between sysPhysMemTop() and sysMemTop() for the applications own use before rebuilding both VxWorks and the boot program.

Setting Upload Path and Mode

1. Click Properties on Collection Conguration Dialog. 2. Change default buffer management conguration if desired.

3. Specify upload path and destination data. 4. Specify upload mode when upload occurs.

Deferred: Upload after collection stops. Continuous: Upload while collecting. Post-Mortem: Upload after system crash.

Tornado Training Workshop

Copyright Wind River Systems Wind River Systems

6-14

In deferred mode, collection continues until either it is explicitly stopped or the event buffer lls. The data may be uploaded after collection stops. This mode has the advantage that upload activity does not appear in the uploaded log. This is the default mode.

In continuous mode, upload occurs periodically during collection as the event buffer lls. The event buffer may be dynamically expanded or contracted during upload. This mode has the advantage that very large logs may be collected, but upload activity will occur in the log.

In post-mortem mode, once the event buffer is lled it starts to be rewritten, in a circular fashion. Event data is only uploaded after logging is stopped, typically by a system crash and reboot. In this mode, the buffer management portion of the dialog changes to specify only the region of preserved memory where the event buffer will be written.

Starting and Stopping Collection

Several methods:
q q

and buttons on WindView control panel. Triggering facility. Programmatic control from target.

For socket upload paths, start evtRecv or open listening WindView graph before uploading.
q

For continuous mode, start the listener before starting collection. wvLib reference page. usrWindview documentation in WindView Users Guide.
Copyright Wind River Systems Wind River Systems

For information about programmatic control, see:

q q

Tornado Training Workshop

6-15

The usrWindview module denes the routines wvOn() and wvOff() which start and stop a typical WindView logging and upload session. The upload mechanism used is the last one of TCP/IP socket upload path TSFS socket upload path File upload path which has been included in the VxWorks image.
q q q

In Tornado, the usrWindview module is implemented in usrWindView.c, together with urWvFileUploadPath.c, usrWvSockUploadPath.c, and usrWvTsfsUploadPath.c, all of which you may nd in the target/cong/comps/src directory.

Triggering
Associates actions with WindView events. Action may be:
q q

Start or stop WindView logging. Call a function. Optionally, disable trigger and/or enable another. Any combination of event type, context, and involved system object ID. Optional conditional expression involving integral global variable or ISR-safe function of no argument.

Trigger res when enabled and event matches criteria:

q

User trigger events, distinct from user WindView events, may be generated using trgEvent().
Tornado Training Workshop Copyright Wind River Systems Wind River Systems

6-16

Using Triggering to start and stop WindView is a good way to collect short event logs focused on the events of interest.

Starting or stopping WindView, or calling a function which you do not explicitly identify as safe to call from an ISR, occurs in the context of a task trgActDef spawned specically for this purpose. In the case of a trigger starting WindView, this means that logging is not started immediately upon the event which res the trigger, but only when the trgActDef task gets a chance to run. For this purpose, then, it is best if you trigger on an event which occurs somewhat before the events of interest. Stopping WindView after an event of interest is an alternative.

The triggering facility is documented in the WindView Users Guide. Triggering may be controlled from the target application using the routines in trgLib; see the reference entry for this library for more information.

Triggering User Interface

1. Click Launch Triggering button on Launch toolbar. This opens the Triggering control panel (below).

2. Click Create Trigger (or Maintain Trigger ) to open the Trigger Maintenance dialog.

3. Edit trigger ring conditions and actions.

Tornado Training Workshop

Copyright Wind River Systems Wind River Systems

6-17

For more information on the Triggering user interface, see the WindView Users Guide.

WindView 2.0
Introduction Conguring WindView 6.3 Events

Tornado Training Workshop

Copyright Wind River Systems Wind River Systems

6-18

Data Collection is based on Events

What is an Event? ...an event is any action undertaken by a task or an ISR that can affect the state of the real-time system. System library instrumentation points. User events.
q q

Examples: Task state change. semGive & semTake. msgQSend and msgQReceive. Timer expiration. Anything you dene as an event.

Tornado Training Workshop

Copyright Wind River Systems Wind River Systems

6-19

The event dictionary, available in the WindView Users Guide, and online via Help => WindView Help => Legend, lists all system WindView events, and describes the conditions under which each event occurs, its effects on task state, and the amount of information logged about the event at each logging level.

User-Dened Events
STATUS wvEvent (eventNo, buffer, bufsize) eventNo buffer bufSize
ID number for the event, which will appear on the viewgraph. address of user supplied data. NULL if none. size of user supplied data, in bytes.

The buffer can contain user data which is passed up to WindView on upload. This is displayed in a hex-dump format. Use this to programatically instrument events in your code.
Tornado Training Workshop Copyright Wind River Systems Wind River Systems

6-20

It is easy to assign bitmaps to user events. This is discussed in section F.4 of the WindView Users Guide. The bitmap les are in the (.bmp) format. They are placed in a subdirectory .wind/windview of your home directory (determined under Windows by the value of the environmental variable %HOME%). The bitmap corresponding to user event N must be named somethinguserN.bmp, where something is anything you like; for example, the bitmap for user event 3 could be named enterFooFunc_user3.bmp.

A useful technique is to create macros for instrumentation functions such as wvEvent(); after development completes the macros can be turned into no-ops so as to avoid instrumentation in the production code.

User Eventpoints
STATUS e (addr, eventNo, taskNameOrId, evtRtn, arg)
Where: addr eventNo taskNameOrId evtRtn address to set eventpoint. an event number for display. the task ID of the task in which to raise the eventpoint, 0 = all tasks. function to call when event point is hit. Event is logged if evtRtns return value is zero. an integer argument to pass to evtRtn above.
Copyright Wind River Systems Wind River Systems

arg

Tornado Training Workshop

6-21

An eventpoint is like a breakpoint, except that instead of stopping the task which hits it, a WindView event may be logged and a function may be called.

The e() function is a WindSh built-in, which also exists as a target function in dbgLib.

Example use:

-> e sin, 10, 0, 0, 0

sets an eventpoint with an ID of 10 at the address of sin() for all tasks. Whenever any task calls the sin() function, WindView user event 10 would be logged.

If the evtRtn argument is used, it should return an integer and take a four-byte argument (passed as an integer). If the returned value is nonzero, the event will not be logged.

WindView Example
Initialization:
Create message queue; Spawn consumer task; Spawn producer task;

Producer Task:
FOREVER { Allocate block; Initialize data in block; Send pointer to block across message queue; Delay one tick; }
Tornado Training Workshop

Consumer Task:
FOREVER { Wait for message (pointer to block) to arrive on queue; Process data in block; Free block; }

Copyright Wind River Systems Wind River Systems

6-22

The full code example is in appendix A.

The Big Picture

Initialization and rst few iterations of producer consumer program.

Tornado Training Workshop Copyright Wind River Systems Wind River Systems

6-23

On this board, INT 6 is the system clock interrupt and INT 3 is the network interrupt. The time scale is in seconds. The system clock is ticking 60 times per second. Tasks are listed in priority order on the left.

The log starts with a network interrupt INT 3, which wakes up tNetTask, which delivers a WDB request to the WDB agent tWdbTask.

Task t4 is spawned by the WDB agent to execute the initialization code. Task t4 spawns tSend1, the producer task, and tRecv1, the consumer task. Eventually t4 is deleted with help from tExcTask.

The tasks tWVUpload and tWvRBuffMgr belong to WindView itself. Task tWVUpload controls upload of event log data from target to host, while tWvRBuffMgr manages the target event buffer, expanding it or contracting it dynamically as necessary. Neither of these tasks is active in the displayed interval.

Zooming In

One cycle of the producer - consumer program. Events are shown in the context in which they occur.
Tornado Training Workshop Copyright Wind River Systems Wind River Systems

6-24

INT 6 , the system clock tick interrupt, occurs during the kernel idle loop and undelays the producer task tSend1.

The producer allocates a block of memory in a semaphore-guarded critical region . It then initializes the data in the block (selected region between user events and ). Then tSend1 sends a pointer to the block across a message queue , and delays itself .

When tSend1 sends the message, tRecv1 is made ready to run, but does not run until tSend1 delays itself . After receiving the pointer to the block from the message queue, tRecv1 processes the data in the block in the interval between user events and . It then frees the block , and pends itself on the message queue waiting for another block to process.

Searching for Events

q Move

cursor over event, and description appears in status bar. q Double-click, and Show Event dialog appears.

Tornado Training Workshop

Copyright Wind River Systems Wind River Systems

6-25

Double-clicking an event, or right-clicking and selecting Show Event... from the context menu brings up the Show Event dialog.

The Show Event dialog allows one to search forwards or backwards in time for events matching particular criteria:
q q

The type of event. The task or interrupt context in which the event occurs. The VxWorks object (semaphore, message queue, watchdog, etc.) involved, if any.

Select or type in the appropriate values, and lock the elds of interest, then click the Prev or Next buttons. For instance,
q

Type user19 in the event type eld and lock only that eld to search for occurrences of user event 19. Double-click on a semTake() event, lock only the object ID eld, and search forwards or backwards in time for a corresponding semGive().

Too Many Events?

There are three means for ltering the data to reduce the amount of information shown.
q

The task pop-up menu can be used to hide any task. Right-click on task name on left side of graph for a context menu. To act on multiple tasks, select Attributes.... Non-executing task states may be hidden by pressing the appropriate button on the tool bar. Specic events and states may be ltered out using the Filter Events/States dialog. Move mouse over event, or double-click on event. Right-click on task state stipple, select Context State Right-click and select Legend to see event dictionary.
Copyright Wind River Systems Wind River Systems

For help remembering WindView symbols, try:

q q

Tornado Training Workshop

6-26

Task States and WindView

The Task State Summary display gives staticstics about the intervals a task spends in different states. Right-click on the task name on the left of the graph and select Information to see this summary. A running task has RUNNING in its state label. It may also have either or both of LOCKED and INHERITED.
q

Preemption locking and priority inheritance are covered in the next chapter.

A task which is not running may be in any combination of the PEND, DELAY, and SUSPEND states. If neither pended nor delayed nor suspended, the task is READY.

Tornado Training Workshop

Copyright Wind River Systems Wind River Systems

6-27

Right-click on a task state stipple line segment and select Context State to see the task state label for that interval, as well as information about when the task entered that state and how long it spent there.

The Task State Summary dialog can display information for the selected interval, the screen width, the full log, or an interval specied by the timestamps of its endpoints.

A task which has specied a timeout in a pending call is in the PEND+DELAY state while pended. It may return to ready if either the timeout expires, or the event it was waiting for occurs.

A nal state, DEAD, is the state reported for a task after it has been deleted. WindView displays no state stipple for this state, but you may see it in the Task State Summary or the Context State dialog.

Memory Events and Analysis Pack

Tornado Training Workshop

Copyright Wind River Systems Wind River Systems

6-28

Analysis packs calculate analog information from event log data. WindView 2.0 comes with two pre-built analysis packs:
q

Memory usage analysis pack exports its results to the graph (pictured above). CPU usage analysis pack calculates and exports a list of CPU idle intervals to a Microsoft Excel spreadsheet. Memory allocations, all types. Logged as memPartAlignedAlloc. Memory frees, logged as memPartFree Memory partition creations, logged as memPartCreate Memory reallocations, logged as memPartRealloc. Additions to memory pools, logged as memPartAddToPool.

When memLib is instrumented, the following events are logged:

q q

Event Base API

Provides C++ and Tcl programmatic interfaces for accessing and analyzing event log data in detail. Provided as a dynamic link library. Library implements objects representing:
q q

Event base as a whole. Task, ISR, or idle context. Individual event. Cursor for iterating through event base. Appendix D of WindView Users Guide Tornado Tools Workshop
Copyright Wind River Systems Wind River Systems

For more information:

q q

Tornado Training Workshop

6-29

The dynamic link library for the event base API is host/x86-win32/bin/apilib.dll (Windows) host/sun4-solaris2/lib/libwvapi.so (Solaris) host/parisc-hpux10/lib/libwvapi.sl (HP-UX) These may be loaded into a tcl interpreter such as wtxtcl using the load command.

Using the Tcl interface is convenient and quick; it is good for one-shot uses or prototyping.

Using the C++ interface yields faster, more structured programs and somewhat more numerically accurate results.

The header le host/include/wvapi.h is available for programs using the C++ event base API.

Summary: WindView 2.0 vs. 1.0

Triggering on events now available. More sophisticated buffer management. The Target Server File System and alternative upload paths. User can hide irrelevant tasks. Analysis packs extend analytical possibilities. Summary data of task states are available on demand. Event data are available via TCL or C++ API for further analysis (the event-base API).

Tornado Training Workshop

Copyright Wind River Systems Wind River Systems

6-30

WindView 2.0 also supports the ability to display logs from multiple processors in the same graph. If the same timstamp source is used on all processors, the logs will be synchronized; no other synchronization mechanism is provided.