Accessing the ENVY repository over a WAN

[Hans-Martin]

Hello,
it is well known that accessing an ENVY repository over a WAN is often not very satisfying, since the ENVY protocol causes lots of round-trip communications which is severely affected by network latency.
Digging around for possible solutions I found that there seems to be a mechanism for accessing a repository via a gateway, but this does not seem to be part of the standard product (see emLibrary>>operationsInterface).

Would this interface be helpful for WAN usage, and is it available to VAST users?

Cheers,
Hans-Martin

[wembley]

Hans-Martin -

The gateway function that you refer to was part of an experiment to determine the viability of WAN-access. It enabled a one-to-one pairing of a developer's image with a gateway image running on a machine "close" (network-wise) to the EMSRV machine. Large-grained messages were sent from the developer's machine to the gateway machine; the gateway machine translated these large-grained messages to normal EMSRV interactions and returned the results to the developer's machine. This code was never intended to be part of the product. I still have a copy of the client code, but I'm not sure I ever had a copy of the gateway server code (or, if I did, I don't now).

If anyone is interested in trying to reinvent the gateway server code and making the gateway work, I would be happy to send them the client code.

[Hans-Martin]

John,
That sounds like it could be a solution for the problem we're just facing:
We do most of our development using windows clients where the EMSRV machine is on-site, so repository access for development is fast.
However, we also have UNIX servers in a data center running images for batch processing. Round trip network latency between data center and the repository is > 100ms, so loading a config map into an image takes several hours
The current hack consists of copying the repository when there's no activity on it, which limits the viability of building images during the day.
I'd be very interested in looking at the gateway protocol. My initial approach would be to write a gateway server in VA which would make it very accessible. I guess it wouldn't be a lot of code, so bootstrapping should not be a problem.
Another possible approach could be to make the gateway completely transparent to the client by splitting the gateway into two processes residing on the server and client machine, respectively:

Code: Select all

+- server machine ---------------+             +- client machine ----------------+
|      emsrv protocol            | GW protocol |            emsrv protocol       |
|EMSRV <------------> GWServer <-+-------------+-> GWClient <------------> Client|
|                                |    (WAN)    |                                 |
+--------------------------------+             +---------------------------------+


(The GWClient could alternatively be running on another machine on the same LAN as the Client)

Cheers,
Hans-Martin

[daswartz]

This "gateway" approach discussion got me thinking.

Here's another approach that might work. What if we changed the VA client code to know about two different envy servers at the same time: a local "cache" server and the remote "master" server. For read accesses the local server would do initial access to the local server, if the object was not found locally, do a read to the remote server. All writes go to the remote server. The cache server is updated with a copy of the remote server periodically. I suspect the vast majority of repository hits are for reads and the vast majority of those reads are on objects which haven't been updated recently ( for a value of recently which exceeds at least 24 hours).

I haven't looked at the code at all yet to determine if this is feasible. But if I remember correctly, ENVY never actually updates an object in the database, it just writes more new objects. That technique could lend itself to this kind of caching.

Doug Swartz

[Hans-Martin]

But if I remember correctly, ENVY never actually updates an object in the database, it just writes more new objects.

Sadly, that's not entirely true. Some kinds of database records (such as disk dictionaries) need to be updated. It's mostly method editions which are never updated. Although this is probably the bulk of database contents, all the read accesses typically traverse the database record structure starting at the root type dictionary and going through other dictionaries which support updating.

For my case (loading a config map) this would not matter much. Exporting/Importing the config map is not an option since that would require versioning it every time (only versioned config maps can be exported). Maybe an easier hack would be to allow export/import of open config map and application editions.

[tc]

Hello,

I've worked in companies with similar issues and in one case, we wrote code to update the master repo with config maps and apps from the local repo.

The update worked for both directions, local changes were written to the master and changes in the master were written to the local repo.

The updating code could run anytime but we ran it around 1 am.

--tc

[Hans-Martin]

Just for fun, I've started writing a simple gateway according to my crude diagram above.
At the moment, it's just one half (the client side) with a cache (i.e., it directly talks to the EMSRV process and caches records fetched from the repository for up to 5 seconds).
So far results are promising: A significant number of requests from the client can be serviced using the cache, which would result in reduced network activity and faster response time when I'd actually use it over WAN. The next step is to build some smart clustering rules and define an extension to the EMSRV protocol to transfer clusters of records.
Figuring out how EMSRV breaks up long responses into 32k chunks took some time

[bpieber]

How is it going with your emsrv gateway? It sounded very interesting.

Cheers,
Bernhard

[Michael Keppler]

I'm not quite sure if that is helpful for you, but our developers do remote Smalltalk development by having a LAN machine running VAST and a laptop with UltraVNC or remote desktop connection to that local machine. That way you get fast round trips for the (local) Smalltalk system and the WAN roundtrip only affects your typing or mouse movement latencies.
We used this very sucessfully to do remote development from Sweden on a repository in Germany.

[bpieber]

Thanks for the tip!

Cheers,
Bernhard