CSWorks: web-based industrial automation

Of CSWorks and software development

CSWorks 1.4.3830.0 released

clock June 15, 2010 21:29 by author Sergey Sorokin

What's new:

  • LiveData Service: open data sources asynchronously
  • LiveData data source providers: improved logging
  • LiveData Manager: response items 'Updated' event arguments
  • LiveData server stack fixes
  • Trend control: multiple minor additions and fixes
  • LiveDataWebServiceDemo sample: learn to work with LiveData Web Service directly
  • AlarmWebServiceDemo sample: learn to work with Alarm Web Service directly
  • Alarm Summary and Trend: support for multiple configurations


CSWorks on YouTube

clock May 26, 2010 23:07 by author Sergey Sorokin

A few CSWorks educational videos are available on YouTube. Watch them in high-definition mode (1280x720):

CSWorks - creating your first application

CSWorks - using third-party OPC servers



CSWorks 1.4.3820.0 released

clock May 25, 2010 21:11 by author Sergey Sorokin

What's new:

  • Server requires .NET 4.0
  • Client requires Silverlight 4
  • Client development: integration with Visual Studio 2010
  • Trend Control: specify date range explicitly
  • Delayed alarms to avoid noise


Trending client performance - Lab 03

clock May 19, 2010 21:18 by author Sergey Sorokin

If you are curious about the number of Trend controls you can run against your CSWorks server infrastructure, you may find this post interesting. We will make minor changes to historical data server configuration, run a few Trend control clients against it and analyze what we see.

Environment

CSWorks 1.2.3800.0
Server: Intel Core 2 Duo @ 2.40GHz, 2 GB RAM, Windows Server 2008
Client: Intel Core 2 Duo T5300 @ 1.73GHz (notebook), 2 GB RAM, Windows XP SP3
Network: Wireless 54 Mbps

Server Configuration

1. As usual, we have to configure History Recorder so it uses some scalable database. Install SQL Server 2008 Express on your server machine.

2. Create database "CSWorks"

3. Create HistoricalData table - see "createCommand" parameter of <dbtarget ...="" name="Standard SQLServer DbTarget"> in CSWorks.Server.HistoryRecorderService.exe.config.

4. Configure SQL Server data source and make it active in CSWorks.Server.HistoryRecorderService.exe.config:

<dbTargetConfig>
  <dbTargets activeDbTarget="Standard SQLServer DbTarget">
    <dbTarget name="Standard SQLite DbTarget" ...
      ...
    />
    <dbTarget name="Standard SQLServer DbTarget"
      providerInvariantName="System.Data.SqlClient"
      connectionString="Data Source=localhost\sqlexpress; Initial Catalog=CSWorks;user id=sa;password=...;"
      ...
    />
  </dbTargets>
</dbTargetConfig>


5. Configure HistoryReaderWebService to read historical data from this database. In the web.config, assign SQLServer target to the primary partition and specify correct connection string:

  <historyTopology>
    <historyPartitions>
      <historyPartition name="partition1" primaryDbTarget="partition1 Primary DbTarget (SQLServer)" secondaryDbTarget="">
        ..
      </historyPartition>
    </historyPartitions>
  </historyTopology>

  <dbTargetConfig>
    <dbTargets>
      ...
      <dbTarget name="partition1 Primary DbTarget (SQLServer)"
                providerInvariantName="System.Data.SqlClient"
                connectionString="Data Source=localhost\sqlexpress; Initial Catalog=CSWorks;user id=sa;password=...;"
                ...
                />
    </dbTargets>
  </dbTargetConfig>

6. Restart HistoryRecorder service and verify that it writes observation to the newly configured database.

Running clients

Before running the clients, make sure you have prepared *.clientConfig in CSWorks.Client.TrendDemo.xap to run from a remote machine. Please see this post for details.

Now run a few Trend clients on your client machine using this command:

start iexplore "http://myserver/CSWorksDemo/TrendDemo.html"

I ran 25 instances, increasing the load by 5-instance chunks.

Results

All 25 clients run without problems, trending data (both live and historical) arrives without delays, server seems to be perfoming fine. Here is a screenshot made on the server machine. Clients consume about 200K of live and historical data every second, server machine uses about 35% of its CPU capacity. SQL Server and ASP.NET worker process are working hard (14% and 18%, respectively) to deliver historical data to the clients.

The spikes in data transfer rates mark moments when Trend control were re-querying bigger amounts of historical data - View->Tracking setting were set to On for all Trend control instances, and all instances were refreshing the whole picture synchronously (well, in chunks of 5 instances, of course).

The spikes in CPU consumption mark moments when I ran chunks of 5 IE instances.

Summary

Not bad for a commodity server box that runs every piece of the deployment: all CSWorks services (LiveData, Alarm, HistoryRecorder), web services and database engine. Potential bottlenecks are:
- the database - not surprising;
- web service layer - but we can scale it out using web farm.



CSWorks 1.2.3800.0 released

clock May 6, 2010 21:54 by author Sergey Sorokin

What's new:

  • History Reader web service: historical data point areas
  • Alarm web service: alarm areas
  • Trend Control: color picker
  • Trend Control: load/save pen configuration
  • Trend Control: line/bar/HLC pens, cursor and tooltips
  • Alarm Summary: edit/load/save subscription configuration
  • Alarm Summary: ack/shelve buttons and hot keys
  • Alarm Summary: sort on-the-fly setting
  • LiveData items and live pens: ScaleFactor attribute
  • Pipes And Tanks Demo: customizable trend control
  • Alarm Demo: passing alarm events to subscribers 


History Recorder performance - Lab 02

clock April 20, 2010 17:09 by author Sergey Sorokin

In the latest release of CSWorks, we have improved Historical Data Server performance. Let's see what History Recorder is capable of now.

Environment

CSWorks 1.2.3730.0
Server: Intel Core 2 Quad Q6600 @ 2.40GHz, 4 GB RAM, Windows 7

Set-up

1. Install SQL Server 2008 Express on your server machine.

2. Create database "CSWorks"

3. Create HistoricalData table - see "createCommand" parameter of in CSWorks.Server.HistoryRecorderService.exe.config.

4. Configure SQL Server data source and make it active in CSWorks.Server.HistoryRecorderService.exe.config:

<dbTargetConfig>
  <dbTargets activeDbTarget="Standard SQLServer DbTarget">
    <dbTarget name="Standard SQLite DbTarget" ...
      ...
    />
    <dbTarget name="Standard SQLServer DbTarget"
      providerInvariantName="System.Data.SqlClient"
      connectionString="Data Source=localhost\sqlexpress; Initial Catalog=CSWorks;user id=sa;password=...;"
      createCommand="..."
      writeCommand="..."
      maintenanceCommand="delete top (300000) from HistoricalData ..."
      maxObservationAge="600"
      writeInterval="1"
      maintenanceInterval="30"
      maxQueryLength="65535"
      queryDelimiter=";"
      writeTxnBeginCommand="..."
      writeTxnCommitCommand="..."
    />
  </dbTargets>
</dbTargetConfig>

Please note we tell History Recorder to keep alarm event records in the database for 10 minutes only, and we perform record cleaning every 30 seconds. This only because our SQL Server is not capable of taking heavy load.

5. Restart History Recorder service. Make sure events are now written to CSWorks database (run "select * from HistoricalData" to confirm it).

6. Using cscript tool, run a script that generates 2000 historical data points:

function main()
{
  var areas = 200;
  var dpsInArea = 50;

  for (i = 0; i < areas; i++)
  {
    var areaId = i.toString();
    while(areaId.length < 4)
    {
      areaId = "0" + areaId;
    }

    WScript.Echo("<!-- area 0000" + areaId + "-AAAA-0000-0000-000000000000 -->");

    for (j = 0; j < dpsInArea; j++)
    {
      var dpId = j.toString();
      while(dpId.length < 4)
      {
        dpId = "0" + dpId;
      }
      dpId = areaId + dpId;
      WScript.Echo("<dataPoint id='{00000000-0000-0000-0000-0000" + dpId + "}' description='Tank fill - " + dpId + "' expression='tank1-" + dpId +"'/>");
    }
  }
}

main();

7. Copy generated historical data point descriptions to RecorderDataPoints.xml:

<dataPoints>
  ...
  <dataPoint id='{00000000-0000-0000-0000-000000000000}' description='Tank 1 fill - 0000' expression='tank1-0000'/>
  ...
  <dataPoint id='{00000000-0000-0000-0000-000000001999}' description='Tank 1 fill - 1999' expression='tank1-1999'/>
</dataPoints>

History Recorder will pick up the change in a couple of seconds and will start saving observation for those 2000 data points to the database. Give our setup some time to stabilize.

Results

After 10 minutes, History Recorder maintains about 3.5 million observation records in the database and writes about 6000 observation records every second on average. Database file size is between 1 and 1.5 gigabytes. Here is a screenshot with Performance Monitor and DbgView windows:

Since our test live data changes in a very predictable way, there is a clear pattern in observation recording on the top perfmon chart. History Recorder memory consumption is under control too. Tracing shows that History Recorder deletes between 120000 and 270000 "obsolete" observations every 30 seconds. As you may have noticed, the maximum number of record it is allowed to delete in one shot is 300K, see 'maintenanceCommand' parameter above. Our setup is properly balanced, so History Recorder does not reach this limit.

If you add more historical datapoints and make total count, say 5000, you may end up in a situation when History Recorder simply cannot write all collected observations in a timely manner, and they will accumulate in the memory buffer. Major symptom will be growing memory consumption by History Recorder. CSWorks 1.2.3800.0 introduces "Write Buffer Size" performance counter that shows current number observations to be written to the database by HistoryRecorder, so this overload scenario becomes more obvious.

Summary

Please plan your historical data management carefully. Use scalable database engine, and give it a lot of spare CPU resources. Use multiple History Recorder machines if needed. If the amount of data is extremely big, use multiple databases and apply partitioning technique described in CSWorks documentation.



Resizable Alarm Summary and Trend

clock April 19, 2010 21:11 by author Sergey Sorokin

Resizable controls - what's the problem?

To save HMI screen space, application developers may want to make some controls resizable. Applying ScaleTransform is not a problem and is a good solution in many cases. See Pipes and Tanks Demo for example - it uses ScaleTransform and redraws screen content when a user changes browser window size.

Unfortunately, ScaleTransform is not an optimal solution when dealing with complex UI elements, especially those containing some amount of text - some pieces may look ugly, some may become too small to be useful. The only solution is to make complex controls able to react to the 'SizeChanged' event and recalculate internal layout at runtime. Latest version of Alarm Summary and Trend Control can do that, and this post shows how application developers can use this functionality.

We will need a container control that lets user change its size, and a piece of code that passes the changes to CSWorks controls. You can develop your own container control (there are a lot of examples available in the web) or use an existing third-party container control for that. I will use RadWindow from Telerik in this example - it serves the purpose quite well.

Resizable Alarm Demo

1. Download Rad Controls for Silverlight trial package from www.telerik.com (DLLs only).
2. Unzip downloaded archive to "C:\Program Files\CSWorks\Demo\Src\tps\Telerik". Telerik DLLs will be unpacked to "C:\Program Files\CSWorks\Demo\Src\tps\Telerik\Binaries\Silverlight\".
3. Open Alarm Demo project ("C:\Program Files\CSWorks\Demo\Src\AlarmDemo\") in Visual Studio.
4. We will use RadWindow control implemented in Telerik.Windows.Controls.Navigation.dll. Add Telerik.Windows.Controls.Navigation.dll and Telerik.Windows.Controls.dll to the reference list for Alarm Demo project.
5. Open Page.xaml file. Add 'telerik' namespace to the UserControl element:

<UserControl ...
  xmlns:telerik="clr-namespace:Telerik.Windows.Controls;assembly=Telerik.Windows.Controls.Navigation"
... >


6. Replace control content with a grid that contains RawWindow that contains CSWorks Alarm Summary, and add SizeChanged event handler for RadWindow:

<Grid>
  <telerik:RadWindow x:Name="radWindow" SizeChanged="OnSizeChanged">
    <telerik:RadWindow.Header>
      <TextBlock Text="Alarm Demo"/>
    </telerik:RadWindow.Header>
    <alarm:AlarmSummary ... />
  </telerik:RadWindow>
</Grid>


7. Open Page.xaml.cs file, implement SizeChanged event handler that adjusts AlarmSummary height and width when RadWindow size is changed: 

private void OnSizeChanged(object sender, SizeChangedEventArgs e)
{
  GeneralTransform gtGlobal = alarmSummary.TransformToVisual(radWindow);
  Point leftTopGlobal = gtGlobal.Transform(new Point(0, 0));

  alarmSummary.Height = radWindow.Height - radWindow.BorderThickness.Bottom - leftTopGlobal.Y - 4.0;
  alarmSummary.Width = radWindow.Width - leftTopGlobal.X * 2;
}


8. Update Loaded event handler so RadWindow shows itself after page is loaded:

private void Page_Loaded(object sender, RoutedEventArgs e)
{
  ...
  radWindow.Show();
}


9. Rebuild the project and run Alarm Demo from Start Menu. Resize RadWindow and watch Alarm Summary changing its layout:

Resizable Trend

 You can do the same to CSWorks Trend control.

1. Open Trend Demo project ("C:\Program Files\CSWorks\Demo\Src\TrendDemo\TrendDemo.Sample.csproj")
2. Add Telerik.Windows.Controls.Navigation.dll and Telerik.Windows.Controls.dll to the reference list.
3. Open MainPage.xaml file. Add 'telerik' namespace as shown above.
4. Change xaml so RadWindow contains TrendControl:

<Grid>
  <telerik:RadWindow x:Name="radWindow" SizeChanged="OnSizeChanged">
    <telerik:RadWindow.Header>
      <TextBlock Text="Trend Demo"/>
    </telerik:RadWindow.Header>
    <t:Trend ...>
      ...
    </t:Trend>
  </telerik:RadWindow>
</Grid>


5. Open MainPage.xanl.cs. Add OnSizeChanged handler:

private void OnSizeChanged(object sender, SizeChangedEventArgs e)
{
  GeneralTransform gtGlobal = _demoTrend.TransformToVisual(radWindow);
  Point leftTopGlobal = gtGlobal.Transform(new Point(0, 0));

  _demoTrend.Height = radWindow.Height - radWindow.BorderThickness.Bottom - leftTopGlobal.Y - 4.0;
  _demoTrend.Width = radWindow.Width - leftTopGlobal.X * 2;
}


6. Show RadWindow on startup:

void MainPage_Loaded(object sender, RoutedEventArgs e)
{
  ...
  radWindow.Show();
}


7. Rebuild the project and run Trend Demo from Start Menu. Resize RadWindow and watch Trend control changing its layout: 




We have gone international: localized Alarm Summary and Trend Control

clock April 13, 2010 20:45 by author Sergey Sorokin

From now on, Alarm Summary and Trend Control can have localized menus, buttons and messages. Norwegian happened to be the first non-English language - thanks to our partner company in Norway.

To set Silverlight application culture explicitly, you can create a 'localized' html file (say, 'MyAlarmSummary0234.no.html')that hosts your application and add 'culture' and 'UIculture' parameters to the Silverlight object. The following piece of Javascript code shows how to instantiate a Norwegian Silverlight application:

Silverlight.createObject("....xap",parentElement,"myPlugin", {width:'100%', height:'100%', ..., culture:'no', UIculture:'no'},{onError:null, onLoad:null }, null, null);

And this is what Alarm Summary looks like:

Update:

Starting from 1.4.3850, there is no more need to edit demo html files to run CSWorks demo application using specific cultures. Use "culture" URL parameter as described in this post.



CSWorks 1.2.3730.0 released

clock April 13, 2010 20:25 by author Sergey Sorokin

What's new:

  • Downsampling performed by History Recorder, Downsampler Service is gone
  • Changes in web service security model
  • Documentation changes
  • Trend control: added right scale
  • Trend control: chart configuration load/save
  • Localized Alarm Summary and Trend Control
  • Signed setup package 


CSWorks Tracing

clock March 17, 2010 20:46 by author Sergey Sorokin

When CSWorks is not working as expected the first thing to do is to see trace messages using a tool of your choice. This can be either some third-party tracing tool or a .NET trace listener.

Using tracing tools

I use DebugView by www.systeminternals.com. You can download the tool for free as part of their SysInternals Suite or as a standalone tool here.

No installation needed, just run dbgview.exe. Please keep in mind that Vista, Server 2008 and Windows 7 users should run this tool with elevated privileges, otherwise it won't be able to catch trace messages. When prompted for a filter, enter "CSWorks":



Make sure you have checked "Capture Global Win32" and "Capture Events" items of "Capture" menu:



Now you are ready to capture CSWorks trace messages. In most cases, this is all you have to do in order to find a problem and take action. The following screenshot shows the case when LiveData Service is denied access to an OPC server:



- when you see this, you may want to run dcomcnfg utility and adjust OPC server permission or run service manager and change the account LiveData Service runs under.

Using .NET trace listeners

You can avoid using third-party software - just turn on .NET TextWriterTraceListener, it is a part of the .NET framework. The only thing you have to do is to modify config file of the service or web service you are troubleshooting as follows:

<system.diagnostics>
  <trace autoflush="true" indentsize="2">
    <listeners>
      <remove name="Default" />
      <add name="myListener" type="System.Diagnostics.TextWriterTraceListener" initializeData="C:\ProgramData\CSWorks\LiveDataServerLog.txt" />
    </listeners>
  </trace>
  <switches>
    <!-- Valid values are Error, Warning, Info, Verbose -->
    <add name="GeneralTraceSwitch" value="Warning" />
  </switches>
</system.diagnostics>

You can read more about .NET trace listeners at Microsoft web site.

Trace level

By default, all CSWorks server components send only error and warning messages, as displayed on the screenshot above. Sometimes this is not enough and you may want to see more details. Then you should increase tracing verbosity level of specific component(s).

For instance, if you need detailed tracing for LiveData Service, edit CSWorks.Server.LiveDataService.exe.config - set tracing level to "Info" or "Verbose":

  <system.diagnostics>
    <switches>
      <add name="GeneralTraceSwitch" value="Verbose"/>
    </switches>
  </system.diagnostics>


After you restart LiveData Service, you will start getting much more output in DebugView window.

If you change tracing level for a web service, edit "GeneralTraceSwitch" value in correspondent web.config file. After you save updated web.config, changes will take effect immediately.