ClarionTest modifications, May 2013

Back when I first wrote ClarionTest, I only ever used it as a post-build task. That is, I'd build my test DLL and then I'd automatically call ClarionTest and load up my test DLL usually with the /run parameter to run all tests automatically. 

Later John Hickey made some nice improvements to ClarionTest, including adding the ability to watch a directory for changes and automatically load the changed test DLL. I never really used that option as the compile-automatically-run-tests-press-ESC-to-get-back-to-my-app cycle worked so well for me. 

And then after I adopted the practice of putting my classes in a separate DLL, I found myself doing a whole lot of testing that involved multiple changed DLLs - the class DLL and the unit tests DLL. My old test cycle wasn't so convenient when I was making changes only to the classes. 

It was time to revisit John's changes. As usual, one thing led to about five others. 

An all-source ClarionTest

A while back I took the radical step of converting ClarionTest from an APP file to an all-source project. I wasn't entirely sure at the time why I was doing it. In part it was a wild impulse to get my hands on the source code in a way not possible as long as it remained in an APP. 

But as I thought about this, I realized that ClarionTest really does belong as source code. It exists because I was fed up with having my business logic buried in embed points, and not only did I want to move that logic into reusable classes I also wanted to be able to test that business logic in some automated way. 

But while the first version of ClarionTest helped me achieve that goal, I soon realized that most of the business logic in ClarionTest itself was embed code. Perhaps that was an inevitable bit of bootstrapping, but once I had ClarionTest it made sense to move the app's own logic to ClarionTest, test it with ClarionTest, and then replace the ClarionTest embed code with the classes. Got it? 

That was step one in the ClarionTest reclamation project.

The more classes I write, and the more I learn about how I can write classes (there is always more to learn), the less satisfied I am with how the ABC class library (the foundation for the ABC templates) does things. That doesn't mean ABC is terrible code - it does lots of things well. But it's a pretty old design, and it does have some code smell. 

So why not make ClarionTest a showcase for all the things that are potentially great about Clarion object oriented programming, separation of concerns, and all those other technologies, techniques and practices that make for better software?

Given that premise, I ripped almost everything out of the ClarionTest source except for the window definitions and the data declarations necessary to compile those windows. And I started over.

Starting over (again and again)

After commenting out all of the existing code and many of the data declarations, I had a bare bones, non-working app. So what were my requirements?

• Be able to load up a test DLL
• Remember the last test DLL loaded
• Mark the individual tests that should be run, or run all
• Display a summary of how many tests have failed and how many have passed
• Option to display only the failed tests
• Fix resizing so the last position/size is remembered

An accept loop

For the last couple of years I've been consulting on a very large Legacy system. And while Legacy is painful to work with, there is one thing I really do appreciate about it: there's an actual Accept loop in the generated code.   I sometimes think it was a mistake for ABC to bury that little bit of Clarion brilliance in the WindowManager.Ask method:

WindowManager.Ask PROCEDURE
  CODE
  IF SELF.Dead THEN RETURN .
  CLEAR(SELF.LastInsertedPosition)
  ACCEPT
    CASE SELF.TakeEvent()
    OF Level:Fatal
      BREAK
    OF Level:Notify
      CYCLE                 ! used for 'short-stopping' certain events
    END
  END

There's nothing in the loop itself except for a TakeEvent() call and the break and cycle options. I think it would make ABC code a lot more readable if procedures still had an Accept loop with those few lines of code. But maybe there are other reasons I'm not aware of. 

Maybe it's nostalgia, and maybe I'll regret not using WindowManager for ClarionTest, but for now I'm going to go with a regular old Accept loop and see where it takes me. If that means coming up with my own WindowManager replacement, I think I'm up for that. 

Resizing

For no particularly good reason I decided to tackle window resizing first. The first step was to find a way to remember the last window size and position.Here I did go with an ABC option: INIClass. 

INIClass is an example of a relatively benign ABC class. Although it conforms to the ABCInclude convention (which I generally despise), and although it's in a file with a bunch of other classes rather than in its own file (which I abhor) it does have the merit of being independent of any other ABC classes. And it provides a convenient way to store/retrieve configuration data. 

I added a relevant global include statement:

INCLUDE('ABUTIL.INC'),ONCE

and a global declaration followed by some global init and kill code:

Settings                                      INIClass                              ! Global non-volatile storage manager
	CODE
    Settings.Init('.\ClarionTest.INI', NVD_INI)                ! Configure INIManager to use INI file
	Main()
    Settings.Kill                                              ! Destroy INI manager

I didn't call my instance INIMgr, which is what the ABC templates use. I called it Settings because that's what the object does - it manages settings. I may be storing those settings in an INI file, but that isn't the defining aspect. 

Complaints: the second parameter to Init is unnecessarily cryptic. I can deduce the _INI part, but what does NVD stand for? And why the need for an explicit call to Kill? Here's the method code:

INIClass.Kill PROCEDURE
critProc  CriticalProcedure
  CODE
  critProc.init(SELF.critSect)
  IF  NOT SELF.Sectors &= NULL
    SELF.UpdateQueue('__Dont_Touch_Me__','Sectors',SELF.Sectors,SELF.Sectors.Family,SELF.Sectors.Item,SELF.Sectors.Type) !***
    DISPOSE(SELF.Sectors)
    SELF.Sectors &= NULL
  END
  RETURN

Blech. There's that infamous '__Dont_Touch_Me__' string that shows up in every INI file. And I still don't see the rationale for an explicit kill - if it hasn't been done at destruct the class should be smart enough to do it. 

Oh well. I'll hold my nose on this one until I get around to writing something that suits me better. 

Here's my Accept loop so far:

	open(window)
	Window{PROP:MinWidth} = 600  
	Window{PROP:MinHeight} = 300 
	ACCEPT
		case event()
		of EVENT:OpenWindow
			settings.Fetch(ProcedureName,window)
        of EVENT:CloseWindow
			settings.Update(ProcedureName,window)
		END
		case accepted()
		of ?Close
			post(event:closewindow)
		END
		
	END
	close(window)

My app remembers its last window position and size but I need to be able to resize the list box that contains the tests and results. 

Again, I resorted to an ABC class: WindowResizeClass. I might want to replace this at some point, but like INIClass it's fairly innocuous. That 'Class' suffix on the name is superfluous - I don't know that I've ever seen a class library where every class has that word in its name. For that matter, ABC is inconsistent - there are a number of class names that do not end on 'Class'. None of them should. 

I had to play around a little with the resizer to get it to do what I wanted.Here's my first cut of the window:

I have some flat buttons in the toolbar, and four controls in the window area: a prompt, an entry field, a lookup button and a list box. The prompt needs to stay fixed, and the entry field should only extend to the right. The lookup button needs to stay fixed to the right side of the entry field. The list box needs to resize to fill the window. 

I ended up with this override of the Init method:

Resizer.Init                            PROCEDURE(BYTE AppStrategy=AppStrategy:Resize,BYTE SetWindowMinSize=False,BYTE SetWindowMaxSize=False)
	CODE
	PARENT.Init(AppStrategy,SetWindowMinSize,SetWindowMaxSize)
	SELF.SetParentDefaults()                                 ! Calculate default control parent-child relationships based upon their positions on the window
	SELF.SetStrategy(?TestList, Resize:FixLeft+Resize:FixTop, Resize:Resize) ! Override strategy for ?LIST1
	self.SetStrategy(?TestDll:Prompt,Resize:FixLeft+Resize:FixTop,Resize:LockSize)
	self.SetStrategy(?TestDll,Resize:FixLeft+Resize:FixTop,Resize:LockHeight+resize:resize)
	self.SetParentControl(?LookupTestDll,?TestDll)

Note the use of SetParentControl to lock the lookup button to the entry field. 

I still have a couple of problems with resizing. I wanted the Close button in the toolbar to stay to the right edge of the toolbar, but the resizer throws an assert if I attempt to use it with a control inside a toolbar. I can see that being a useful restriction for height changes, but in fact you can change the position of controls inside a toolbar so I chafe at this restriction. I got around it with a line of code inside the Resized event handling. 

Another problem is that my entry field isn't resized as fully as I'd like. As I make the window wider the entry field stretches but at a slower rate, so that its right edge gets further from the right side of the window instead of keeping a constant distance. 

I wouldn't be the first person to write my own resizer class, but again I can live with these limitations for now. 

Down the wrong rabbit hole

I had bit of a detour early on. In reviewing the original code I saw that there was a call to read the list of test DLLs from the directory. First there was a queue declaration:

AllFiles                                    QUEUE(File:queue),PRE(FIL)            !
											END                                   !

and then some code to load up the queue.

	Free(AllFiles)
	DIRECTORY(AllFiles,CLIP(Set:FolderToMonitor) & '\*.DLL',ff_:DIRECTORY)

Simple enough, right? Silly to make that into a class, right?

Nah, turns out I have one on hand already that adds a few bells and whistles. Time to make it part of DCL!

That was easier said than done. My directory class, which also handles things like creating and removing directories, relies on several other classes which had to be brought in to DCL. In their previous ClarionMag incarnation I used some different naming standards and some of the include files had also changed. And then I had a problem with the Clarion CreateDirectory and RemoveDirectory functions, which confused me for a bit as the class has methods by those names also. I do use the Clarion functions as well and it turned out I had to add cwutil.clw to the project so the code would get compiled. 

I still have a problem with the directory class - one of the unit tests fails on an attempt to create a directory and then remove it when it isn't empty. But I'll have to leave that for another day. As an aside, one of my goals is to modify ClarionTest so I can execute a set of test DLLs and log the results - that will give me a view of which if any tests are failing throughout the system. 

So that was a lot of work just to replace a few lines of code, but it's all part of getting more ClarionMag code into DCL, so it's worth the effort. 

Except that I didn't really need it after all, at least at this stage of the rewrite. The purpose of the directory listing was to show all of the test DLLs. I don't generally work with more than one test DLL at a time so there's no real need to display that information on screen.

Moral: Don't spend time on code that isn't needed! Although as noted it was probably time those classes made their way into DCL. 

Loading up a test DLL

One line of code lets the user look up a DLL with FileDialog:

FILEDIALOG('Chose a test dll',TestDll,'Test DLLs|*.dll')

And two calls at the appropriate places save and restore the last test DLL:

settings.Fetch(ProcedureName,'TestDLL',TestDll)

settings.Update(ProcedureName,'TestDLL',TestDll)

This is all pretty standard stuff in ABC apps, so even if you're writing procedural code, Clarion is constantly writing OO code for you. 

Now, how about loading up the test DLL? Really this just takes two lines of code:

			TestRunner.Init(TestDllName)
			TestRunner.GetTestProcedures(ProceduresQ)

TestRunner loads up the test procedures into a predefined queue that's part of the ClarionTest classes. I am going to want to do some formatting of the queue, which raises an important question about where those kinds of changes should be made. If I want to use my TestRunner's queue for display purposes then I'm going to need to add some UI-related fields into the queue (for colors, styles, etc). That's inevitably going to make things messy and will blur the lines between the display of testing information to the user and the code that actually execute tests. 

I decided pretty quickly to follow the approach previously used in ClarionTest, which was to create a separate display queue in the application itself, and copy the relevant information over from the queue populated by the TestRunner object. That's a bit of extra work, but it keeps UI separate from business logic and it means I don't have to make changes to my class library to handle UI improvements. 

Here's the LoadTests routine with TestsQ as the UI queue and ProceduresQ as the list of test procedures:

LoadTests                               ROUTINE
	do SetDirectoryWatcher
	free(TestsQ)
	logger.Write('Loading tests from ' & TestDllPathAndName)
	str.Assign(TestDllPathAndName)
	TestDllDirectory = str.SubString(1,str.LastIndexOf('\'))
	if TestDllDirectory <> ''
		if not exists(TestDllDirectory)
			message('The test directory ' & TestDllDirectory & ' does not exist')
		ELSE
			setpath(TestDllDirectory)
			TestDllName = str.SubString(str.LastIndexOf('\') + 1,str.Length())
			TestRunner.Init(TestDllName)
			TestRunner.GetTestProcedures(ProceduresQ)
			logger.Write('Found ' & records(ProceduresQ) & ' tests')
		END
	END
	loop x = 1 to records(ProceduresQ)
		get(ProceduresQ,x)
		IF ProceduresQ.TestGroupName <> PreviousGroupOrTestName
			PreviousGroupOrTestName = ProceduresQ.TestGroupName
			CLEAR(TestsQ)
			TestsQ.GroupOrTestName = ProceduresQ.TestGroupName
			TestsQ.GroupOrTestLevel = 1
			TestsQ.ProcedureQIndex = 0
			TestsQ.GroupOrTestStyle = Style:Bold
			ADD(TestsQ)
		END
		TestsQ.GroupOrTestStyle = Style:Default
		TestsQ.GroupOrTestName = ProceduresQ.TestName
		TestsQ.GroupOrTestLevel = 2
		TestsQ.ProcedureQIndex = x
		TestsQ.TestResult = ''
		TestsQ.TestResultStyle = Style:Default
		TestsQ.Mark = false
		Add(TestsQ)
	END

The extra ADD at the top half of the loop is to handle test groups, e.g.:

Directory monitoring

With most of the core functionality in place, I still needed to re-implement directory monitoring which uses the DCL_System_Runtime_DirectoryWatcher class. The original class was written by Alan Telford (building on work by Jim Kane) and is described in Clarion Magazine article Waiting For Files With Clarion Threads. I've made a few minor changes, including moving the suggested event trapping code into a TakeEvent() method which you place in the Accept loop. Then you just need to derive the DoTask method which will be triggered any time a file in the monitored directory changes. 

Initially I run the unit tests any time there was a directory change, but this had two problems. First, I was getting a number of events each time I built a new test DLL and copied it into my UnitTests directory. I really only wanted a single event. My initial solution is a bit of a hack: I set a variable called TimeOfLastDirectoryChange whenever a change is detected, and then I use the following code to delay the test execution until I've had a quiet time of somewhere around a second (the value of DelayBeforeAutoRun is configurable via INI file). 

	if TimeOfLastDirectoryChange > 0
		if clock() < TimeOfLastDirectoryChange or clock() >  TimeOfLastDirectoryChange + DelayBeforeAutorun
			TimeOfLastDirectoryChange = 0
			do RunTests
		end
	end

What I'd really like is a way to handle this inside the class itself, but I haven't come up with a clean solution yet. 

The other problem is that the directory watcher is indiscriminate - it triggers when any file changes. One of my unit tests updates an INI file, and each time the INI file is updated the tests are triggered, so ClarionTests sits in a loop executing all the selected tests every second (provided directory monitoring is enabled). 

Clearly I need to filter out only the DLL changes. I'm not sure of the best way to do this, although if nothing else I may yet use my directory class. 

Also I'm not too impressed with the code I'm using to manage the on-screen list of test procedures.  I need to add icons and some more sophisticated marking code, and I can see that becoming a mess in a hurry. 

Still work to be done

There's always still work to be done. Besides monitoring for DLL changes only, I need to find some way of presenting summary information and showing only, say, failed tests. The UI is pretty ugly, so if you have some suggestions for how it can be improved please pass them along. Oh, and not all the buttons work yet. Some of that may have to wait until next week. 

Meanwhile, ClarionTest is now free of any and all third party dependencies. While that has resulted in some loss of functionality (at least over the short term) it does mean that it's now much easier to build ClarionTest. You don't need to download or register any additional templates. Just load up the cwproj and compile. 

As always you can download the latest and greatest DCL including ClarionTest at GitHub.