Hello world!

Welcome to WordPress.com. This is your first post. Edit or delete it and start blogging!

Posted in Uncategorized | 3 Comments

Come and meet me at Expert Days 2010!

On the week of the 22nd of November you are welcome to come and see me at the Israeli Expert days 2009 (http://www.expertdays.co.il/ ) in Ness College in Tel Aviv.

I will be giving two one-day workshops:

Everything you ever wanted to know about MSMQ 4.0

Windows Azure – Microsoft Cloud

The workshops will be given in Hebrew, but I would switch to English if any student will ask for it.

I can also give the same workshops on demand in your place.

See you there!

Yoel

Posted in Uncategorized | Leave a comment

PDC – Day 3 – Boku, .NET services and more on Azure queues

Day 3 keynotes were about Microsoft research, presented by Rick Rashid. Rick’s presentation was probably the most impressing presentation in PDC – even I did not know how large the Microsoft research team is (850 researchers – larger than most universities), and the breath of projects they are involved in – from software proofing, to weather control, health research (yes!) and surface programming. However to me as a father (and as a former LOGO instructor) the most impressing project presented is Boku – a programming environment for children. If you have children (and / or you are still, like me, a little bit of a child yourself) go see it.

After Boku, returning to the technical stuff was a bit hard…

I went to see Clemens Vasters‘ presentation on .NET services. Clemens showed how you can connect two applications through the cloud – even if they run on public Internet connections, corporate environments, NATS etc. To enable that, Microsoft is delivering a whole bunch of new bindings for WCF – the "relay" bindings – each corresponding to an existing "normal" binding (for example, BasicHttpRelayBinding corresponds to BasicHttpBinding. There is no MsmqRelayBinding yet – it may be a good exercise to develop one… According to Clemens, one of the reasons that Microsoft did not come up with such binding is the difficulty of keeping end-to-end transactional messages protocol through the cloud. However, I think it is a good idea anyway to avoid transactional messages.

Later in the day, I went to see Sriram Krishnan’s talk on best practices of programming on the Cloud. Sriram is an excellent presenter and I highly recommend that you go see his talk (should be available online on the PDC site in a few days) – especially if you are distributed applications architects  – not only those developing for the cloud. Sriram preaches for stateless, loosely coupled and simple solutions (did I tell you I hate transactions?). He strengthened my conclusion that cloud programming to the enterprise programmers is what VB.NET is to VB6 programmers – it looks the same, but it is actually quite different – and it will take time to adjust and think "the cloud way". You can see more in his site.

I learned a little more today about Azure queues. The interface to these queues is HTTP REST interface (which implies that duplications are part of life and there are no real transactions – but you know that already). Here are the APIs:

Queues:

Similar to get properties in MSMQ, except it also returns the message count (do I hear you clap?) and the other properties are free form name / value pairs.

Messages:

In the URI, you can also ask for the number of messages you want, and also specify a timeout. If you do not delete the message within the timeout, it will be returned to the queue.

Same as GET, but the message remains visible in the queue (you knew that already, didn’t you?)

"lightweight commit" You have to call that after GET – otherwise the message will be put back in the queue

Deletes all the messages in the queue (purge).

The Azure samples (provided with the SDK) wrap the REST methods with a .NET class, RestQueue, which itself is wrapped with a class called… Ahm… MessageQueue! Isn’t that fun? You already know that when programming with System.Messaging on a widows form you have to explicitly put the namespace for Message, now you also have MessageQueue in a different namespace. Should be fun.

That’s all folks… four queue methods, five message methods. Is that all we need? I guess that in most cases, the answer is surprisingly YES. I would love to hear your views.

Technorati Tags:
Posted in Uncategorized | Leave a comment

PDC – Day 2 – some more on Azure queues

Day 2′s keynotes where about Windows 7 – which apparently not going to include lots of bells and whistles (unless you are totally excited of the ability to re-order icons on the task bar), but hopefully will be faster and more stable, so people will migrate to it from Windows XP (and enjoy MSMQ 4.0 features, of course :-)).

I attended Brad Calder’s "Essential Azure storage services" and learned some more on Azure’s queues:

In general, queue is one of the three "native" storage object types in Azure (if you are not using SQL services) alongside with containers (containing blobs) and tables ("poor men’s database").

An Azure’s queue supports the following methods (exact names and parameters TBD):

  • Create
  • Clear
  • Delete
  • Get length (isn’t it nice to have this as a first-class method? :-))
  • Enqueue (Message)
  • Dequeue(timeout) –> returns a message. After the timeout, the message is returned to the queue unless explicitly deleted.
  • Delete(message)

A message can be up to 8K in size (that said, the real data will most likely be elsewhere – in a container, table, or database.

I kind of like the Enqueue / Dequeue / Delete paradigm. As devoted readers of this blog know, I always wanted some way to do "lightweight" transactional receive from a non transactional queue, and Azure provides just that (BTW, there is also a way to achieve that in MSMQ 4.0 using sub-queues – I may write a blog about that sometime). The queuing is far from being as feature-rich as MSMQ is, but I can definitely see scenarios when it will be the mechanism of choice. It worth following it up.

That is for today… Universal Studios visit was fun. Looking forward for tomorrow.

Technorati Tags:
Posted in Uncategorized | Leave a comment

PDC – Day 1 – the Azure day & yet another queuing

I guess that you all already read in the press or elsewhere about the new bomb Ray Ozzie had thrown to the keynotes hall in the PDC, declaring Microsoft’s new "operating system" – Microsoft Azure.

Azure is actually called "operating system" just because this is what Microsoft is used to call its products :-). In fact, this is a completely new beast – the move of Microsoft from selling the platform to owning & leasing it. At least at the first stage, Microsoft will not sell the new OS, but rather lease storage, bandwidth etc. on its own servers – what will put it as a competitor to companies like Rackspace, Godaddy, and, well, Google. In corridor talks I had, this was the most controversial aspect of the new move – Microsoft became such a great company because it had developed open systems (well, not open source :-)) and developed and echo-system – in contrast to the close approach of Apple computers. In the new move, Microsoft want to take the OS & tools alongside with the actual hosting & maintaining of the data – just as Google is doing with Google Apps. It is interesting to see if it will continue that way, or eventually sell the Azure as OS and bring along partners.

So far for business thoughts… On the technical side, Azure is a really exciting platform for easily hosting services, that brings "hosting" to a new level – from just storage to a complete set of enterprise-grade services.

In an "Azure Overview" talk I attended right after the keynotes, Munuvir Das describes that Azure to services is like OS to applications – it frees the developer for thinking of the ugly details of deployment and hosting. Indeed, the demonstration of a few clicks deployment of distributed application looks quite impressing. However, it is clear that Azure also requires the developer to learn a whole new set of tools and paradigms, just like VB programmers could not just "evolve" to VB.NET programmers…

Among other things, Azure presents yet another queuing (if you are not already confused enough between MSMQ and Message Broker). Azure queuing looks like the main paradigm Azure programmers will use, in order to allow the system to perform heavy background tasks and be a little more than ASP.NET++. One first observation from the code I saw (mainly in  Steve Marx "developing your first Azure application" presentation) is that Azure queuing had taken a "light transactional" approach – you can "GetMessage" – which give you the message exclusively – and when you are done, you should explicitly delete the message from the queue. If you fail to do so, the message will return to the queue after a timeout. There are not full transactions or distributed transactions. I kinda like this approach actually. I should devote some time to play with Azure queuing and come with some more solid conclusions…

This is all for now – I am almost late for 2nd day keynotes – see you soon!

Technorati Tags:
Posted in Computer and Internet | Leave a comment

PDC, Day 0 – pre-conference WCF session

Most people think PDC starts at Monday the 27th, however for me (and for ~1000 more people :-)) PDC started at Sunday with the pre-conference.

I went to hear Juval Lowy and Ron Jacobs speaking on WCF. Juval has a quite radical view on WCF – WCF is no less than the .NET killer and the new way to write every application in the world (well, maybe except high end graphics and device drivers). I must say his argument is quite convincing argument – with WCF you have logging, timeouts, performance counters & thread management out of the box, and this provides a step forward from .NET, exactly as .NET was a step forward from COM / C++.

Juval presented an interesting study – in the average enterprise solution, about 3% of the time is dedicated to solve the business solution, while 97% is dedicated to "plumbing" – messaging, connectivity, logging, security, etc. etc. This ration tends to worsen over time, as most of the maintenance time is also dedicated to plumbing problem.

I am not sure I totally agreed with the conclusion, that using WCF can make programmer 20 times more productive (I guess that even with WCF the time dedicated to plumbing does not go away so easily). However I definitely agree that doing "plumbing" right can be the difference between successful and failed project.

Ron presented REST services over WCF, which seems like a Microsoft attempt to make some peace with the web community and save some bandwidth in the way. Interestingly Juval presented WCF as something far larger than "web services platform" (as Microsoft’s "official" claim) while Ron showed the right way to work if you anyway want to use WCF for mere web services :-).

There were about 500 participants in the session (I did not know that so many people are willing to give up Sunday just to hear about WCF!). Interestingly, a vast majority of the audience was from overseas. I am sad to say that the ratio of males to females in the audience was worst than the ratio of plumbing to business code in Juval’s study (out of the 500, I think I spotted two or three women in the audience). Is infrastructure programming a male only profession? I find that hard to believe…

Takeaways:

  • Take your infrastructure (plumbing) code seriously
  • WCF is much more than just web services, but if you are building web services, look at WCF+REST
  • Look for building all your code using WCF (preferably with MSMQ binding, of course :-))

Overall a great session. I set for eight hours with no intermission except lunch and loved every minute.

Looking forward to the rest of the PDC….

Posted in Uncategorized | 2 Comments

Meet you in PDC 2008 + USA Visit + Is Send() thread safe?

It is hard to believe, but three years had passed since PDC 2005… The buzzwords then where WCF, WPF, LINQ & Vista. This time it is going to be cloud services, Oslo & Windows 7. Actually the "old" stuff that was presented in 2005 still looks a bit new for most developers (myself included  :-)). It would be interesting to see how fast all the new stuff will catch up.

I will come to the USA on the 22nd of October and leave at the 6th of November, and in-between I will be available for consulting (except, of course, the PDC itself – 27-30 of October). If you are interested in a one day (or more) of review of your MSMQ solution, or MSMQ training in your site – just drop me an email to yoel@msmq.biz . If you are coming to the PDC, I will be very happy to meet you and have a chat on MSMQ, distributed solution or any other subject – again, just drop me an email.

I wouldn’t like to end this post without giving you some technical value :-) . Well, recently two customers asked me why MessageQueue.Send is not thread safe. Indeed, MSDN documentation states (in "MessageQueue Class" documentation):

"Only the following methods are safe for multithreaded operations: BeginPeek, BeginReceive, EndPeek, EndReceive, GetAllMessages, Peek, and Receive."

Obviously, Send is not in the list… This looks a little weird, because MQSendMessage C++ API is definitely thread safe.

This question was raised in the MSMQ newsgroup about three years ago, and I have to admit I did not fully answer it… This time, I decided to do my own little research, and wrote the following program:

class Program
{
    static MessageQueue outQueue;
    static void Main(string[] args)
    {
        outQueue = new MessageQueue(@".\private$\mtQueue");

        for (int i = 0; i < 100; i++)
        {
            Thread thr = new Thread(new ThreadStart(MyThreadProc));
            thr.Start();
        }
    }

    static void MyThreadProc()
    {
        Message msg = new Message();
        for (int i = 0; i < 100; i++)
        {
            msg.Label = string.Format("{0} : {1}", Thread.CurrentThread.ManagedThreadId, i);
            outQueue.Send(msg);
        }
    }
}

(Note that .\private$\mtQueue creation code is not included – you can create it in advance using MMC).

As you can see, the program sends 10,000 messages using 100 thread. The program runs beautifully with no failure. Does it mean that MessageQueue.Send is always thread safe? Not necessarily…

Looking at MessageQueue.InternalSend code (using Lutz Roeder’s Reflector) reveals the following code:

private void SendInternal(object obj, MessageQueueTransaction internalTransaction,
                                        MessageQueueTransactionType transactionType)
{ <Some code deleted>
              Message cachedMessage = null;
              if (obj is Message) { cachedMessage = (Message) obj; }
              if (cachedMessage == null)
              {
                           cachedMessage = this.DefaultPropertiesToSend.CachedMessage;
                           cachedMessage.Formatter = this.Formatter;
                           cachedMessage.Body = obj;
              }
   <Code for sending cachedMessage using MQSendMessage>
}

Where is the problem, then? If you are using a Message object (as I did in my code) there is no problem – it is sent "as is" and as long as you do not use the same Message object in all your threads it would work.

The problem starts when you want to use the .NET "shortcut" for serializing and sending an arbitrary .NET object in MessageQueue.Send. In this case, the .NET code will use DefaultPropertiesToSend.CachedMessage – a member variable of MessageQueue, for doing the serialization. Since this member is the same in all threads, you will get an exception when using "Send" in multiple threads.

So, I changed MyThreadProc() in the original sample as follows:

    static void MyThreadProc()
    {
        for (int i = 0; i < 100; i++)
        {
             outQueue.Send("Hello, World!");
        }
    }

Bingo! I got an exception:

Unhandled Exception: System.InvalidCastException: Specified cast is not valid.
   at System.Messaging.Interop.MessagePropertyVariants.Lock()
   at System.Messaging.Message.Lock()
   at System.Messaging.MessageQueue.SendInternal(Object obj, MessageQueueTransaction internalTransaction, MessageQueueTransactionType transactionType)
   at System.Messaging.MessageQueue.Send(Object obj)
   at TestMultithreadSend.Program.MyThreadProc() in C:\Projects\TestMultithreadSend\TestMultithreadSend\Program.cs:line 32
   at System.Threading.ThreadHelper.ThreadStart_Context(Object state)
   at System.Threading.ExecutionContext.Run(ExecutionContext executionContext, C
ontextCallback callback, Object state)
   at System.Threading.ThreadHelper.ThreadStart()

To sum up – Send is actually thread safe, as long as you always send a Message object and never use send a .NET object directly. Using the Message object, BTW, is always a good idea – since it lets you add label, timeouts, recoverable option and all this stuff that make your MSMQ solution a real enterprise solution.

Close to the end of my small research I found that this issue was actually addressed recently by Microsoft – you can find more about it here .

All the best – hope to see you in PDC,

Yoel

Posted in Uncategorized | 5 Comments