Machine Performance Check - A Useful Tool for Machine QA?

MPC Performance Values

An MPC report currently lists 24 different performance values, plus the individual offset values for all MLC leaves.

We describe what these values mean (in the order as they appear on the screenshots), together with the thresholds which are currently not user-definable.

The description is based on the Reference Guide (see literature below).

All measurements are based on acquired kV and MV images.

The imagers are at 150 cm during acquisition.

Isocenter

Size (Threshold: ± 0.50 mm)

The size of the treatment isocenter is defined as the maximum distance of a beam's central axis from the idealized isocenter.

It has therefore the meaning of a radius, not diameter.

The central axis is defined by the center of rotation of the MLC.

The treatment isocenter is defined as the ideal intersection point of the beam's central axis over a full gantry rotation.

This involves image acquisitions on eight representative gantry angles every 45 degrees.

MV and kV Imager Projection Offset
(Threshold: ± 0.50 mm)

Once the treatment isocenter has been determined, it can be projected on the imagers.

The imager projection offset is the maximum distance of the imager center from the projection of the treatment isocenter.

It is a measure of the correctness of the IsoCal calibration.

A low value is important for matching. A low kV offset value is essential for CBCT image quality.

Beam

In contrast to the Geometry Check (which is only available for 6 MV), Beam Checks can be performed for all photon (including FFF) and electron energies.

A jaw collimated 18 x 18 cm field is set and an MV image is acquired.

The central 13.3 x 13.3 cm are used for the evaluations.

These are strictly relative, which means that one of the checks has to be set as baseline, and results are relative to the baseline.

Output Change
(Threshold: ± 2.00 %)

It is the average percentage of change in detector response in the central part of the detector.

Uniformity Change
(Threshold: ± 2.00 %)

It is the total percentage of change in detector response in the central part of the detector.

It captures the worst-case, the variation between the two imagr pixels with the lowest and the highest ratio.

For this evaluation, high frequency noise is filtered.

The check can indicate changes in the target/filter system, as well as changes in beam steering.

Center Shift
(Threshold: ± 0.50 mm)

This is the relative shift of the field center, defined by the jaw collimated field, with respect to the baseline.

The field center is established through detection of the jaw edges on the MV image.

The shift represents a summary value on the precision of the beam steering system, the collimation and MV imaging system.

Collimation

The positioning accuracy is determined at gantry 0°. The position of the collimating devices is defined as the point perpendicular to the edge which shows the steepest gradient.

MLC (maximal, mean, individual offset)
(Threshold: ± 1.00 mm)

The positioning accuracy of each leaf is determined using a static comb pattern with alternating leaves.

It is measured as the distance of the MLC leaf tip from the MLCs center line.

The center line is defined as the line through the center of rotation of the MLC that is perpendicular to the edges of the leaves.

On the HD120, leaf 2-59 (out of 60) is evaluated for each bank.

Jaws Offset
(Thresh. X1/X2: ± 1.00 mm) (Thresh. Y1/Y2: ± 2.00 mm)

Jaw edges are detected on a symmetric 18 x 18 cm field. The result is measured as the distance of the jaw edges from the center of rotation of the MLC.

Rotation Offset
(Threshold: ± 0.50°)

Five images of an MLC comb pattern are acquired at collimator angles -90°, -55°, 0°, +45°, +90°.

The rotation offset is the maximum deviation of the nominal versus the actual collimator rotation angle observable through the edges of the MLC leaves.

Gantry

Absolute
(Threshold: ± 0.30°)

The absolute positioning accuracy is defined as the coincidence of the couches vertical axis with the central beam axis at gantry 0°.

For a movement along the vertical couch axis MPC evaluates any lateral or longitudinal shift of the phantom with respect to the beam and the treatment isocenter as the absolute gantry angle positioning error.

Relative
(Threshold: ± 0.30°)

The angle of the gantry is evaluated as defined by the MV imaging system using the geometric phantom.

The relative positioning accuracy of the gantry is the maximum offset between the angle determined by the MV imaging system, and the nominal gantry angle.

The values are compared for eight representative gantry angles (0, 45, 90, 135, 180, 225, 270, 315)°. Since the individual results may be positive or negative, the relative positioning accuracy is also signed.

Couch

The positioning accuracy of the different couch axes is measured with respect to a reference position. Upon MPC software installation on a TrueBeam system, the Varian service engineer acquires couch reference.

Lateral
(Threshold: ± 0.70 mm)

Describes the positioning accuracy of the lateral couch axis on a 5 cm travel range (IEC61217 positive direction, to the right).

Longitudinal
(Threshold: ± 0.70 mm)

Describes the positioning accuracy of the longitudinal couch axis on a 5 cm travel range (IEC61217 negative direction, away from the gantry).

Vertical
(Threshold: ± 1.20 mm)

Describes the positioning accuracy of the vertical couch axis on a 15 cm travel range (IEC61217 positive direction, upward).

Rotation
(Threshold: ± 0.40°)

Describes the positioning accuracy of the patient support angle on a 10° travel range (IEC61217 positive direction, view source to couch: CCW).

Pitch and Roll
(Threshold: ± 0.10°)

The pitch and roll axes of the PerfectPitch couch are measured in isocentric coordinates. This means that the mechanical travel ranges of the other axes will be different from the above values.

The pitch axis is rotated by the mechanical maximum of 3° ((IEC61217 positive direction, rear end of couch lowering).

The roll axis is rotated by the mechanical maximum of 3° ((IEC61217 negative direction, view towards gantry: CCW).

Rotation Induced Couch Shift
(Threshold: ± 0.75 mm)

The center of rotation of the couch is determined through a motion on all available rotational axes and differs from the location of the treatment isocenter. The rotation-induced couch shift is the offset of this center of rotation from the treatment isocenter.

Evaluation Accuracy

In the MPC reference guide, Varian distinguishes between algorithm accuracy and measurement accuracy:

The accuracy of MPC evaluated parameters is determined by the total measurement accuracy including data acquisition as well as processing.

In order to provide number for the expected accuracy two scenarios are distinguished:

1.) The accuracy of the image processing algorithms (algorithm accuracy) is assessed by using MV/kV image series generated based on a computational model of the treatment machine containing all components relevant for MPC. This procedure allows for deliberately introducing deviations of the individual components and for assessing the resulting accuracy.

2.) The total measurement accuracy is estimated as standard deviation of the MPC performance values based on daily acquired MPC data over a period of two weeks.

Commissioning problems on TrueBeam 1275

While software installation and commissioning of MPC ran without problems on TrueBeam 1469, the MPC MV images on the second TrueBeam were only garbage:

According to Varian, this can happen on certain machines, and MPC cannot be used (note that clinical MV images were more or less fine, with the exception of a vertical bar in the left half of the image, which could not be removed by calibration).

After the IDU on TrueBeam 1275 was replaced, MPC could finally be commissioned on the machine.

The first successful check on TrueBeam 1275 was performed on April 13, 2015:

Literature

Machine Performance Check Reference Guide, Version 1.1. P1008852-001-A, August 2014. Available via my.varian.com

 

As part of the TrueBeam 2.0 MR1 update in March 2015, Varian installed the Machine Performance Check software on our systems. At a single press of the beam-on button, TrueBeam performs a series of beam & geometry checks,

and immediately displays the results:

(Two MPC Offline Client screens with some results of the 6X Beam & Geometry Check.)

The idea is not to replace an established machine QA program, but to complement it. Varian recomments that the 6X Beam & Geometry Check is performed on a daily basis by the therapist, or even directly before a demanding treatment (SRS, etc.).

Check Preparation

MPC uses the same phantom as the IsoCal calibration/verification procedure. But it neither uses the table extension to connect the phantom with the couch, nor any accessory for the collimator slot. To positon the phantom on the couchtop, a dedicated bracket is used which goes into the H2 slot.

MPC is integrated into the TrueBeam software. It is started via the Major Mode screen. After loading the "6X Beam & Geometry Check", the user has to drive the couch and the imaging arms to the starting position on the dedicated keyboard.

Performing the Check

Beam On is only pressed once, then the user may sit back and watch. The software automatically rotates the gantry, the collimator, drives the MLC, moves the couch, pitches and rolls it, and acquires a series of MV and kV images. The complete show takes about five minutes.

As soon as a check is complete, the results are displayed on the right TrueBeam monitor. If all goes well, a green icon appears. If not, the check may be repeated.

The Geometry Check is currently only available for 6 MV photons. Beam Checks may be scheduled for all photon and electron energies:

Evaluation of Check Results

Checks can be evaluated either online (on the TrueBeam console) or offline. Two PCs may be nominated for installation of an Offline MPC client. The offline client has read-only access to the checks which are mirrored to the I-drive via PeerSync.

As soon as a check is completed, it shows up in MPC History. The color coding is nearly self-explanatory. The grey icons are the baselines:

When the mouse is hovered over the name of any of the performance values, a small icon appears right to it, which may be clicked. This opens a plot which shows the history of the selected parameter.

For 6X, we want to see the trend of the Beam Output Change, and click on the icon:

The plot shows that beam output was stable over the last few days, and then suddenly dropped (vertical grey bars appear whenever a note is added):

The reason was an intentional dose output recalibration. As follow-up to an absolute dose measurement on April 9th, TrueBeam beam output had been reduced by entering the calibration factor of 101.7% in service mode. The following MPC Beam Check correctly detected the change in dose output due to the recalibration. In this case, we appointed the most recent (failed) 6X Beam Check as new baseline, and signed it off.

The next screenshot (not from MPC) shows on the right side the absolute dose measurement which had led to the 101.7% recalibration factor for 6X. The recalibration also left its mark on the next morning check, as measured with QUICKCHECKwebline (left):

If Something Goes Wrong ...

... the first thing to try is to repeat the check. Sometimes, "nonsense images" are acquired during a beam check,

which on our machine most frequently happens for 6 MeV electrons, but also for other photon or electron energies. Usually, the repeated acquisition is fine.

If a beam is not properly tuned, dosimetry interlocks may stop the beam:

We see this behavior from time to time for 6FFF (but not 10FFF), so it is probably a beam tuning issue on one machine. Acknowledging dosimetry faults and repeating the check usually helps.

The CCHU interlock, our best friend since the PerfectPitch upgrade a year ago, also sometimes shows up in MPC.

There also exists a warning level for the check results. If a value is still within thresholds, but close to outside of thresholds, the check icon turns orange:

In the check of April 14 on TrueBeam S/N 1469, it was the size of the isocenter which triggered the warning level:

The warning level is defined as approaching 10% of the threshold's limit:

How the size of isocenter can be reduced is a different question. It is also not quite clear what the meaning of a "negative size" is.

On the other machine (TrueBeam 1275), isocenter size is smaller:

Discussion

Can MPC make the physicist's life easier? Theoretically it could, if we were allowed to replace some established QA procedures by using MPC.

Take for instance the performance value "isocenter size": it has never been easier to determine the size of the isocenter sphere. However, our current mandatory QA standard (ONORM S5290-1) requires that the size of the isocenter shall be determined for ALL PHOTON ENERGIES, not just for 6X. Don't ask why.

We currently evaluate which items of the MPC Geometry Check also would comply with our QA standards.

Of course, MPC can always be used as an "add-on check". In this sense, it is very useful and has a huge potential, if one considers the low release number (MPC 1.0 MR1). However, we already perform several "add-on checks" on a daily basis (Catalyst, PicketFence, 3D/3D-match for PerfectPitch couch, touch guards, dry run, etc. etc.), because we follow some vendor recommendations and other user's experiences with the TrueBeam system. This goes far beyond the mandatory checks described in the QA standards.

So in order to keep the overall QA time within acceptable limits, we will probably have to reevaluate the usefulness of some of the current QA checks before we add MPC as "another add-on check".

Update, April 30, 2015: after optimizing workflows, we were able to add MPC to the morning check without increasing the total time requirement. MPC is now performed on a daily basis (6X beam & geometry check).

Update, July 6, 2015: MPC Version 1.1.10 was installed together with TrueBeam version 2.5 last week. Isocenter size on #1469 is still too large if one compares it with installation product acceptance (IPA):

During IPA in 2013, the measured size was 0.273 mm (IsoLock value for gantry and collimator axes).

Update, Sep 3, 2015: MPC does not run stable with FFF beams. This is a known issue finally confirmed by Varian engineers. As a consequence, Beam Check of FFF energies fails most of the time. There is currently no solution.

During daily 6X Beam & Geometry Check, MPC hangs frequently on both machines. In such a case, the check has to be started from the beginning. All data and images acquired up to this point are lost.

Without any intervention (except for periodic IsoCals), isocenter size on #1469 dropped. Values of #1275 and #1469 are now very similar. Interesting, isn't it?