[openmapi.org devel] Understanding OpenMAPI architecture and rationales

[Florian von Kurnatowski]

Hi Johannes,

thanks for your response - I hope it's somewhat valuable and  
meaningful for you and also the community to discuss things in such a  
way; for my own projects, I usually gain a lot when people outside the  
core team try to understand design decisions or even challenge them.

> Right. While there are several protocols built on top of MAPI there is
> not one (open) one that is actually capable of implementing the MAPI
> API efficiently.

Is there a list of shortcomings that e.g. Microsoft's Exchange WebDAV  
access protocol has? I haven't looked at it in great detail, but  
obviously it's used by Microsoft's Entourage client on the Mac, and  
that seems pretty complete, functionality-wise, and also exhibits nice  
real-time behavior for notifications, etc., when used against an  
Exchange server.

I understand that the request/response nature of http-based protocols  
like WebDAV may create a challenge for things like notifications, on  
the other hand it seems that things like ActiveSync can quite  
efficiently get to a "push" experience using long-lived http  
connections.

> Furthermore the protocol we are using is based on
> ONC-RPC/XDR, which is an open standard, defined in some RFCs. It is
> also very small and efficient, by the way and works well on mobile
> platforms.

What are the firewall properties of this? GIven more and more devices  
live on a non-VPN Internet connection, especially when it comes to  
mobile devices that are possibly even living inside carrier clouds  
with restricted connectivity (some GPRS/3G APNs offer cheaper  
connectivity if only http is used through a carrier-provided Proxy),  
and given people are trying to access their email from within other  
organisation's internal networks (e.g. when at customer sites), every  
protocol used that is not http-based (or has an http-binding) should  
be carefully evaluated these days.

Furthermore, afaik, ONC-RPC is request-response as well, so how are  
notifications handled? Is the proxy server establishing connections to  
the client system or are there some long-lived connections involved or  
are notifications polled, and if so, at what kind of interval?

> Actually the cost is reduced significantly. The proxy already works
> and has debugging features built in. Backends only need to implement
> the NMapi interface which is much easier, obviously, than implementing
> a network protocol in addition to the API. The proxy/NMapi will also
> be able to provide caching, performance improvements, etc.

Caches, in caching theory, produce benefits if they are at least an  
order of a magnitude (e.g. 10x) faster than the backend they are  
caching for. This is because quite obviously, keeping the cache in  
sync with the back-end has substantial cost and effort associated with  
it, too; we've had to learn that lesson at Scalix, where, for a while,  
we over-cached, and through all the issues that came with it, at the  
end of the day, weren't able to realize the expected benefits of  
various caching layers, so we had to redesign and actually scale back  
on the caching.

Can you therefore elaborate what kind of situation this "cache" is  
supposed to cover and how performance would be improved by it?

Also, how would overall performance improvements be achieved? It seems  
that in some backend situations, the NMAPI API would be wrapped around  
something that is already client-side anyway, e.g. in the case of  
Scalix it seems that the current work being done wraps the NMAPI  
element around our Windows-Side Outlook MAPI connector. This talks to  
the Scalix server using the proprietary (actually it is documented  
publicly in the old HP OpenMail version) UAL protocol. On top of this  
we would now see the NMAPI and Proxy, with another network-type ONC- 
RPC connection to the actual OpenMAPI client. Just based on the depth  
of this protocol and technology stack, I have a hard time seeing how  
this would perform any better than a solution, where the server would  
start offering a "MAPI protocol" (as per above) and the Open  
application integration side would only consist of a library used by  
the applications and implementing the client-side of this protocol.  
Again, from a development point of view, probably not much difference,  
but it still seems to me that the proxy actually adds to the overhead.

The additional benefits - logging/debugging/etc. could well be  
realized in the client-side library. I understand that a runtime  
component is slightly more powerful and centralizing it means that it  
doesn't need to be updated/pushed to all the client application nodes,  
but again it's performance, scaleability and complexity I'm worried  
about.

Thanks again,
Florian