COSH.EXE

A COM Shell for TCL running on Microsoft NT

by

David Shepherd at ICR GmbH, Ueberlingen, Germany

 
 

COSH is a shell programm for TCL like WISH - in fact it *is* WISH with the addition of a COM interface called ITclScript. Perhaps you could call COSH the "COM version" of WISH. 

For anyone involved in distributed processing on Microsoft NT systems, COM is the first word in the vocabulary. It turns out that COM and TCL can work together quite happily and can actually compliment each other in a couple of  important respects. Some points to note about COSH are: 
 

  • COSH differs from WISH in that it can be remotely started on any machine via a call to the WIN32 API "CoCreateInstance". It does this by implementing a COM "Class Factory"
  • You can write in scripts at the COSH console, or you can pass in scripts for evaluation via the ITclScript interface. This lets you remote start plain-vanilla COM shells and then pass in a bootstrap script for evaluation. This makes COSH a good wrapper programm for server-style scripts running in a distributed environment under NT.
  • There is a separate version of COSH (coshd.exe) which is built on the standard IDispatch interface, so you can start and call into COSH instances from COM automation clients such as Visual Basic and Delphi. Thus in Visual Basic you could, for example, start a COM shell instance on any machine, then construct a Tcl script in a VB string variable and cause it to be evaluated in the remote COSH interpreter.
  • When programming components designed for things like  Microsoft Transaction Server or Active Directory Services, Microsoft has mandated that these be packaged as COM components. COSH (or an equivalent in-proc server, aka DLL) can function as a generic COM component and thus can leverage Tcl code directly into the Microsoft distributed processing mechanisms. 
  • It is also easy to install and use. 
 
 

Availability

Binaries and sources are available for download at 
cosh07.zip
http://home.t-online.de/home/dshepherd/cosh07.zip

COSH Installation 

COSH needs TCL 8.0 to run and VC++ 5.0 to compile 

The following files need to be installed on any machine which will run COSH. On the server side they are: 

  •  COSH.EXE          The TCL shell programm (like WISH) 
  •  COSHPS.DLL       A "Proxy/Stub" DLL used to remote client calls to COSH 
  •  COSHD.EXE        Same as COSH, but built on IDispatch 
  •  COSHD.TLB        A Type Library needed by the IDispatch interface 
  •  COSHREG.TCL      Registration script 
You can install these files to any directory you like. Copy the files there and then run the script COSHREG.TCL from that directory. This will create the relevant registry entries needed by COM to remote start the shells. 

In addition there is on the client side, as a demonstration 

  -  COSHCLIENT.DLL   A TCL extension DLL demonstrating a COSH client 
                      which calls the ITclScript interface directly (vtable). 
                      See below for details. Also note that the "eval" function 
                      in ITclScript can be called by any COM automation client 
                      via the standard IDispatch interface in coshd.exe. 
 
 
 

Why bother with COM? 

Microsoft has gone all out over COM and has successively built larger portions of COM and OLE into the operating system itself. For anyone working on programming distributed systems on NT, COM is *the* method of choice to get servers and clients connected and components managed, because you are using native OS services to do it. 

But while COM of itself, being a relatively elegant solution for component interoperability, is a Good Thing, Microsoft has built a services superstructure on top of COM (OLE, ActiveX et al) of enormous complexity. If, as a software developer, you subscribe solely to the Microsoft Way, you already have your work cut out for you - its reputation for being hard to learn and use is thoroughly deserved. 

Fortunately, COM/OLE is not an all or nothing proposition. You can also say "as much COM as necessary and as little as possible". 
 
 

A distributed scenario 

COSH comes out of a project for a system of distributed server processes which has some similarity to workflow systems. A requirement of the system is that it can be controlled from a single point-of-administration. This central administration program can create and monitor a number of server processes on various machines, can perform load balancing among them etc etc. The servers read from and write to message queues and can be arranged into "processing networks". 

This product runs on Windows-NT and has a modular component architecture, so COM is the first thing which springs to mind. However, we also have a significant investment in TCL scripting technology and, seeing the amazing flexibilty which it affords us, we want to retain TCL as far as possible as the glueing technology of our products. While we do not track each and every Microsoft technology as and when they are released, we definitely want to use such services as DCOM, Transaction Server, Message Queue, Active Directory and so on. On the other hand, we like to package our components as TCL extension DLLs and scripts - a nice, easy, effective and lo-tech way of doing things. What to do? 

In the end, the idea of a COM-capable TCL shell distilled out of what we wanted to do. We would install on each machine in the network (1) TCL/TK plus a number of support packages and (2) a generic TCL shell which can be remotely activated and can accept scripts for evaluation from its client. The "client" in this case is a central administration and monitor programm, displayed as a "Game Board". This program creates COM shells as needed according to its load balancing rules and bootstraps them as server processes by pumping scripts for evaluation into them at load time via the ITclScript interface. This is nice because all the server scripts can be kept in one place. 
 
 

What is the current status of COSH? 

On a scale where 0.1 represents "almost useless" and 1.0 is "production quality" COSH is about at 0.70. Plenty still to do, but on the other hand you can do at the moment about 90% of all things you need to do with it. 
 
 

The ITclScript interface 

Disregarding for a moment the standard COM functions such as QueryInterface, AddRef and the like, this interface contains exactly one function, called "eval". It is prototyped in IDL like this: 

    HRESULT eval ([in] BSTR script, [out] BSTR* result,  
                  [out] long* retcode, [out] BSTR* errorinfo);  
 

It is pretty obvious what this function does. It takes an arbitrary script and evaluates it in its host interpreter, returning the result, the status code (TCL_OK, TCL_ERROR etc) and maybe the errorInfo stack trace. 
 

The CoshClient Extension DLL 

A TCL extension DLL called CoshClient.dll is part of the distribution. It demonstrates how to remote start, terminate and feed scripts to a COSH program. It makes use of the vtable interface in COSH.EXE and uses COSHPS.DLL to marshall the call parameters across process boundaries. It instantiates an object of type COMSHELL. It implements the command: 

     comshell ?machine-id?  

When called without the machine-id argument, the command attempts 
to create a COSH instance on the local machine, as specified in the 
relevant registry entries. When machine-id is specified in UNC 
or IP form, it tries to create a remote instance, using the remote 
registry. 

If successful, the command returns a token (cosh000, cosh001 etc) which 
is itself a command. Use this command as follows: 

     cosh000 eval script  

     Synchronously evaluate $script in the COM shell. If an error 
     occurs on the remote machine, it is propagated to the local 
     machine and the remote "errorInfo" call trace is added to the 
     local one. 
 

     cosh000 release ?-lockserver?  

     Release our reference on the interface. This will normally cause 
     the server to shut down. Add option -lockserver to keep the 
     server running. (Performs an additional AddRef on the interface). 
     The server shuts down by internally calling the "exit" command. 
     The token is thereafter invalid. 
 

     cosh000 getptr  

     Get the actual pointer value of the Interface. It is returned 
     formatted with "%08X". Formatted back to a 32-bit value, you 
     could pass this pointer on to other COM aware programs and 
     reference the COM shell from there. (That programm would need 
     to call AddRef on the interface pointer) 
 
 

ITclScriptDisp Dual Interface 

COSHD.EXE implements the same function "eval" as described above, but builds on the standard IDispatch interface instead of using a custom proxy/stub DLL to do the call marshalling. If you wish to connect to a COM shell via this interface, you must instantiate an object of type COMSHELLD. 
 
 

COSH Activation options 

At present COSH is hardcoded as a "Single Use" server. That means each time an interface is instantiated with a call to CoCreateInstance, a new and independent COSH is started. This precludes connecting to an already running server. 

In the source code there is a define (REGISTER_ACTIVE) which will activate code which enters the current instance of the COSH server in the Running Object Table (ROT). Any client which from then on requests an instance of ITclScript on that machine will receive a pointer to that running instance. 

One nice thing here is that COM will serialize multiple client calls so that no problems arise with the non-multithreadedness of TCL. 

If you need to mix those models - ie sometimes connect to a running instance and sometimes create a new one on a single machine, then it looks at first glance as if you are out of luck. This is because, when you connect via COM, you are connecting to an *interface*, and that interface has only one definition on any one machine (which you can find in the registry under HKEY_CLASSES_ROOT\\Interface\\<Interface_ID>). I suspect, however, that it will not be too difficult to find some kind of workaround for that. 
 
 

The call to "eval" in ITclScript is synchronous 

Of its nature, COM only deals in synchronous calls. When you start to need asynchronous calls, callback notification, exception propagation and what have you, things start to get very hairy very quickly when you have to do things the Microsoft Way. You need to start creating interfaces which, although defined in the server, are actually implemented in the client. (Or was it the 
other way around?) Your client has more or less no choice than to run multithreaded, which creates a whole new dimension of hassles and snags for what are, for the most part, otherwise relatively simple server and client programms. 

Thus there is no provision for callback mechanisms in the ITclScript interface - all calls are synchronous and that's all there is to it. 
 

However, TCL would not be TCL if there were no way around that. 

I would like to mention in this connection a mechanism we haved used to achieve the same end. It involves using not COM as the RPC mechanism but the TCL-DP package. Our problem was, from a single point-of-administration to start and monitor various servers in a distributed system, preferably without resorting to the use of launch daemons. We use an ITclScript client to remotely 
start the COSH shells on the various machines. Then that client programm passes in the scripts necessary to bootstrap the servers via the ITclScript interface. 

The client code looks like this: 
 

  set csh [comshell 156.98.0.61];# start remote shell - return token 

  #====== read the server bootstrap script from local file  

  set f [open server_script.tcl r]  
  set script [read $f]  
  close $f  

  #======= pump server script into comshell using token from above  

  $csh eval $script  
 
 

Next we tell the comshell to create a DP server socket on any available TCP port and to return the port and ip-address needed to connect to it, to which the client then connects. 
 

   #====== start dp server in the remote COM shell  
  
   set svr_addr [  

       $csh eval {      ;# eval this script on the remote machine  

            package require dp             ;# needs dp from Zeno  
            set connect [dp_MakeRPCServer] ;# make server on any port  
            lappend retval [fconfigure $connect -myIpAddr]  
            lappend retval [fconfigure $connect -myport]  

       }  
    ]  

    #========= retrieve server address and connect  

    set ip [lindex $svr_addr 0]  
    set port [lindex $svr_addr 1]  
    set sock [dp_MakeRPCClient $ip $port]  
  

At this point we have two different connections open to the COM shell - a DP socket and the COM connection. The two connections compliment each other in the sense that, while in DP you can only call already existing procs in the server (either synchronously or asynchronously), the ITclScript interface allows you to evaluate abitrary scripts in the server, (albeit only ynchronously). In practice, you can create the procs via ITclScript, which you then call asynchronously via DP. 

If you don't need to evaluate any more scripts in the server but only calls its procs, then you can release your reference on the ITclScript interface. Normally this would have the effect of terminating the server, but by calling "AddRef" on the interface you can lock it into memory. The code below does this (see the CoshClient code). 

    #====== lock server and release our COM interface  

    $csh release -lockserver  ;# release our reference with addref  
  
  

You can then do asynchonous RPCs using the dp_RDO command together with the -callback option. 
 
 
 

DCOM is Not Easy 

The thing I like most about COSH is that it keeps my COM involvement to an absolute minimum while still opening a lot of doors on to Microsoft technologies which are, after all, not *that* bad.. In doing so, I don't need to slavishly follow the Microsoft Way and learn the arcane details of the IConnectionPointContainer or IParseDisplayNames interfaces. One interface with one function is about as basic as you can get in COM - and there are good reasons to keep everything very straightforward when working with DCOM. 

DCOM is definitely wierd. For one thing, when you remote start a process on another machine, it has no window. It only shows up in the Task List. Programms with windows, such as TK, still function OK though. I don't know why this is or if you can (or should want to) do anything about t. If you start it on the local machine though, it has a window, but only if you specify the UNC name of the computer or no name at all - if it is started locally via its ip address, it again has no window. 

I have experienced other wierdnesses too. For instance, I have two machines networked. I install COSH server and client on both. Client A with Server B works fine, but not Server A with Client B. (I suspect "Service Pack" discrepancies). 

Or, a Dispatch interface which works perfectly well on the local machine will give strange error messages when called remotely ("Stub received corrupt data" and the like). And they told me DCOM was just COM "with a long wire" - don't you believe it! 

Sometimes COM seems to hang for about 5-10 seconds or so before completing a remote function call - again, don't know why. 

As a COM server, COSH participates in the NT Security schemes. This may cause unexpected hiccups during development - it certainly did for me. Beware of security concerns, permissions and the like 
 

The code in COSH has been hacked from different sources - maybe there are bugs, although everything seems straightforward enough. Maybe there are just kinks in my network setup - it is not very homogeneous. Maybe everything will be fine in NT 5.0. Who knows? DCOM can be a real pain. If you experience DCOM problems - well I am as much in the dark as you. 

First test locally, then remotely. You can connect to a running instance of COSH with a debugger. 
 

I recommend "Professional DCOM programming" by Richard Grimes (Wrox Press) as a DCOM-manual and a good introduction to COM in general. 
 
 
 

What's to do? 

  • DLL inproc version 
  An in-process version of COSH might be quite useful when integrating TCL extension 
  DLLs into services like Microsoft Transaction Server. Still have to figure out a good 
  way of getting the bootstrap scripts evaluated in the DLL at load time. 
  • Combine COSH.EXE and COSHD.EXE into one executable 
  Strictly speaking, only one interface (a dual interface) should be necessary to 
  enable both IDispatch and vtable bindings. However there always seemed to be a gotcha 
  where that wouldn't work and in the end I gave up and made two separate versions. 
  For instance, the WIN32 API "LoadRegTypeLib" insists on overwriting registry 
  entries which effectively disables vtable binding. This causes at minimum a 
  performance hit and at worst hard-to-debug crashes. 
  • command line switches ROT, safe interp etc 
  COSH could benefit from a couple of command line switches which influence the startup 
  behavoir - Safe interp or not. Enter the instance in the Running Object Table etc. 
  Also specify security attributes. 
 
  • Auto package installer 
  Might be interesting to give COSH the ability to accept arbitrary data, which it writes 
  to file at the remote end. This could be the basis of a mechanism which can remotely 
  install DLLs and scripts on the remote host if they're not there already. 
  • "Security" 
  The thought of evaluating any script on any machine is probably a nightmare for those who 
  are security conscious. Still, as a COM server COSH is covered by the "NT security blanket" 
  - whatever that means - and TCL itself offers some handles for implementing security policies 
  through safe interpreters. Also TCL-DP from Zeno, which we use extensively in connection with 
  COSH, offers further ways of limiting the scope for mischief. If anybody could offer advice 
  about which mix of security mechanisms offer the best coverage, I would be pleased to hear 
  about it. 
 
  • more friendly COM error reporting 
  • testing testing testing 

Good Luck! 

 -- 
o David Shepherd 
o Independent Software Author 
o T/F (+49) 7557-91015 
o mailto:dshepherd@t-online.de 
o http://home.t-online.de/home/dshepherd