Getting started with Python & Selenium

For those coming from Microsoft world. Python tool sets are available to develop in Visual Studio. Recently I was working on a situation where I have check a particular website every couple of hours. I did not know at I have to do it for some time, so first time I did not manually and then second time when I did, I know I have to code it. I remember someone told me Python is available in Visual Studio so I thought I will try python in Visual Studio.

Install:

First thing we need to do is to install python tools. When you start a new project in Visual Studio, pick python and if it is not installed, it will ask you to install python tool set and acknowledge it and install them. It normally take couple of mins to install them.

Now that we have python installed, open VS again and create a new python project. We normally use PIP to install python packages if we were using traditional python programming. We need to do the same in VS environment as well. Here is the stackoverflow question and answer. I followed the first answer and able to install the packages I wanted.

There is a possibility you might find ‘pip’ in there, make sure you follow this comment

@cal97g When the panel is smaller, you just have to change the dropdown menu from “Overview” to “pip”. I just expanded it in the picture to show it all at once

Now that we have pip. Select ‘install selenium’ and that’s it. It will install selenium and we are all set to go.

Selenium:

Now that we have all the pre-requisite. Let’s jump in and start coding. For my testing, I started out with the example from selenium-python documentation. I used the code as it is to see how it works before I start experimenting with it.

from selenium import webdriver
from selenium.webdriver.common.keys import Keys

driver = webdriver.Firefox()
driver.get("http://www.python.org")
assert "Python" in driver.title
elem = driver.find_element_by_name("q")
elem.clear()
elem.send_keys("pycon")
elem.send_keys(Keys.RETURN)
assert "No results found." not in driver.page_source
driver.close()

I used the above code the chapter 2. Now with no other changes, copy the code in the python file and try to run it. There is a possibility you will run into this error

“selenium.common.exceptions.WebDriverException: Message: ‘geckodriver’ executable
needs to be in PATH.”

We can solve this by getting the latest version of geckodriver from here and adding it to the system PATH. Before you continue, restart the VS so that it has latest path.

Lets try again, now if the path was picked up properly, it should open the firefox. In case you run into an error like this

“Message: Expected browser binary location, but unable to find binary in default location, no ‘moz:firefoxOptions.binary’ capability provided, and no binary flag set on the command line”

It is because it does not find FireFox in the Path. It can be solved couple of ways. One way is use explicit path of Firefox in the code like the example here. But I like the solution of adding firefox.exe to the system path.

 

Deploying Yeoman generated app in Azure.

Finally I decided to take a plunge into cloud by signing up with Azure. I have been working on SPA application using AngularJS. I decided to try hosting the application in Azure. There are lot materials out there on how to build and deploy SPA into Azure. I am really impressed with the documentation and help available out there to accomplish this task. Since I haven’t written any blog in a year, I thought I will take this opportunity to write a blog about my experience and few things I found along the way.

Build a web application using Yeoman:

I am a lazy programmer so if there are things that make my development easy I will take that route. To get started, I took Yeoman’s help on generating the seed project so that I do not need to worry about the directory structure etc., Yeoman help wire up everything for me. There are ton help on the web, just in case if you haven’t used it. Not to repeat the whole steps, here is one of the site which explains how to get started.Before you go any further, make sure the starter application is working locally by running

grunt serve

If the starter page doesn’t show up then you have problems with the yeoman setup and fix the problem before you proceed.

Source code repository:

Next is source code repository. I decided to use bit bucket for this task. it is free and for me it serve the purpose incredibly. I highly recommend to check it out, in case if you haven’t heard or use it. It has excellent tutorials on how to add an existing project and create new repo.

Hosting:

Next stop is Azure. Create your account and login into Azure portal.Here is a good video on how to add a new web app in Azure from Azure Fridays. Follow the video, you will be able to create a new web application in seconds.

Now that we have all the infrastructure and code ready to go, lets take about how do we go about deploying. Deploying is little tricky at least for me because I was using Web Storm for my development (not visual studio). Here are few things I had to do to make this process automated.

Modify deployment Script:

When you create SPA you will end up using ‘npm’ and ‘bower’ packages. We need to make sure, when we deploy all those packages will be installed in the azure. This is the first hurdle. Azure uses Kudu engine for deployment. So we need to add two files to our project to enable azure deployment and they are

.deployment

deploy.cmd

in the root directory of the project. Even for that there is help. Follow this link and it will help you create those files. The script, it generated will add all the npm packages (as of this writing) but not bower, so you need to modify the deploy.cmd to add bower packages during the deployment. Open deploy.cmd and add the following lines after “::3 npm install” if condition.

:: 4. Select node version
IF EXIST "%DEPLOYMENT_TARGET%\bower.json" (
  pushd "%DEPLOYMENT_TARGET%"
  call bower install
  IF !ERRORLEVEL! NEQ 0 goto error
  popd
)

This will help Azure deployment to add both npm and bower packages in the web site.

You will not deploy the whole file structure Yeoman created as the web site since it has lot of things they are intended for development like testing. You do not want to distribute that code to production and also you will want do all the goodies like linting, minification and uglyfying and trim down the code to small distribution package. Fortunately for us, grunt helps us with this. To do that, we run the following in cmd line.

grunt build

This will create a new directory ‘dist‘ in the directory structure this is what we need to deploy to azure. Couple of ways we can deploy this and I went the the route of creating a branch for deployment and setup continuous deployment from that branch. But remember since Bower packages will not be installed by default we need make sure bower install runs along with deployment that means we need modified deployment file and also the json files distributed along with the ‘dist’ directory. So I added four highlighted statements to the copy task before deploying the dist subtree to branch.

// Copies remaining files to places other tasks can use
    copy: {
      dist: {
        files: [{
          expand: true,
          dot: true,
          cwd: '<%= yeoman.app %>',
          dest: '<%= yeoman.dist %>',
          src: [
            '*.{ico,png,txt}',
            '.htaccess',
            '*.html',
            'images/{,*/}*.{webp}',
            'styles/fonts/{,*/}*.*'
          ],
        }, {
          expand: true,
          cwd: '.tmp/images',
          dest: '<%= yeoman.dist %>/images',
          src: ['generated/*']
        }, {
          expand: true,
          cwd: 'bower_components/bootstrap/dist',
          src: 'fonts/*',
          dest: '<%= yeoman.dist %>'
        },
          { cwd: '', src: "deploy.cmd", dest: 'dist/deploy.cmd' },
          { cwd: '', src: ".deployment", dest: 'dist/.deployment' },
          { cwd: '', src: "bower.json", dest: 'dist/bower.json' },
          { cwd: '', src: "package.json", dest: 'dist/package.json' }
        ]
      },
      styles: {
        expand: true,
        cwd: '<%= yeoman.app %>/styles',
        dest: '.tmp/styles/',
        src: '{,*/}*.css'
      }
    },

That is it, with all these steps now I can easily build and deploy to azure easily.

If you think there is a better way to do this, I would love to hear from you.


			

Debugging infinite loops with Visual Studio

Once in a while we run into a situation where the program run into infinite loop. Best symptom is the application seemed to be frozen. There are lot of different ways to approach it. One simple way is to put as much log as possible and dump it to a log file and have a UDP log viewer like log4net viewer and see what is happening. I like this approach, I call it layback debugging. This requires you need to know where this loop is happening and add as much log as possible. By the way there is nothing wrong with having more logs, but remember to make sure production is not noisy with all these logs. There are other brute force debugging methods like message boxes, putting break point and walking the code etc.,

Recently I ran into a situation where one of our Silverlight application fezzes after some time, there is no special pattern so we do not know exactly where it is happening but we had a ball park idea. To identify the problem area I used Visual Studio rather than logging or message box approach. So in our example, I am going to debug a Silverlight application which might be freezing after some work.

To do this, start the application and keep running it till it freezes. Now launch Visual Studio and open the solution of the application.

Goto Debug -> Attach to process (make sure ‘Show process in all sessions’ is checked) This will open a window and there scroll down and identify the process that you want to attach debugger to. In our case type will be ‘Siverligt’

Now that the frozen application is attached to the debugger, we can put break point and find out where the application is failing. But we do not know where is the current process is. If we do not know where the current process is how do we go about find out where the execution is? The answer is break at the current execution line, this can be achived by

Going to Debug -> Break All, this will stop at the current executing line but it will not stop the execution of the program, which means everything you could do in debugger mode is available to you at this time and you can resume execution by pressing F5 (Run). I would recommend not to use the first line where the program might be as the culprit causing it to go in the loop, rather, press F5 (Run) again and let it run and then break all again. See where is application breaks, keep doing it couple of times, this should give you a very good idea which part of the code might be the culprit for the infinite loops. The beauty of this is, it gives you the call stack as well, so you can see how the execution got to that state as well.

Visual Studio has ton features to help debug code and if you want to learn more about how to use these tools and features, head to Microsoft Debugging in Visual Studio.

Hope this help somebody. If there are better ways of debugging or anything related to this subject please send me a note, I love to hear it.

Windows Phone over Starwars MMO

After a long wait, I was finally able to login and play Star wars MMO, I was so psyched. I logged in and created my first Sith toon then I logged out since I have to go and learn and do some cool stuff in Windows Phone 7, how about that? So here I am at Microsoft and started my Windows Phone 7 application that I wanted to develop for some time. I never get time to do Windows Phone since I was doing few other things in my spare time. So here are the things, I got recommended by Microsoft to get started on Windows Phone development and thought I share it with you all and as a book mark for my future reference.

Code Samples

Silverlight Toolkit

“How do I” videos

Design Resources

Download free e-book for Windows Phone 7

Chirs is hosting the event and he has a fantastic blog and resources for Windows Phone development. I have been using Jounce for all my Silverlight development so obviously I decided to used Ultralight.MVVM. I keep you all posted on my progress on the application that I am developing.

Component One Silverlight FlexGrid Tip 5 (Group Summary Row positioning)

Continuing on from the previous tip on grouping, we go one step further. When we generate the grouping, Component One provides couple of options to how to render them. Well there are 3 to be precise. Component One have a property called ‘GroupRowPosition’. Which lets the developer to position the group row on top of the group or below the group or don’t show the group altogether.

1. GroupRowPosition=’AboveData’

<c1:C1FlexGrid Name="flexGrid" ItemsSource="{Binding PeopleCollection}" AutoGenerateColumns="False" GroupRowPosition="AboveData" >

As the name suggests, on UI rendering, C1 will place the group summary information above the data as shown below

image

If you do not provide this option in the grid, this is the default behavior.

2. GroupRowPosition=’BelowData’

<c1:C1FlexGrid Name="flexGrid" ItemsSource="{Binding PeopleCollection}" AutoGenerateColumns="False" GroupRowPosition="BelowData" >

This forces the summary row to show at the bottom of the group row. This is similar to Excel, when you group, you have an option to render the summary row at the bottom.

image

3. GroupRowPosition=’None’

<c1:C1FlexGrid Name="flexGrid" ItemsSource="{Binding PeopleCollection}" AutoGenerateColumns="False" GroupRowPosition="None" >

Not sure why you would do that but it is available. With this option the group row is removed but the grouping is still in place.

image

If you are requirements needs lots of grouping and want to create grouping and look and feel like Excel, please have a loot at C1FlexGridExcel control which is derived off of C1FlexGrid control. Which by fault look like Excel.

Thoughts on Unit Testing

Yesterday I had a great conversation with one of my friend about few things one of them was Unit testing. He is a big proponent of Unit Testing. Good unit testing (see I did not write more unit testing) could  help identify and resolve the bugs in the code very early. This also helps code maintenance. I use unit testing where ever and when ever possible. I will not sit and break my head over how will I do unit test when I develop form over data patterns. On the flip side when you develop a program that involves some kind of logic that manipulate data then for sure you need to identify the components and test it.

This was one of my friend example, why would we need unit testing, lets assume _a and _b are variable and instead of doing if (_a == _b) if you would do it, if (_a = _b) then you have problem. If you would write an unit test for it, you will catch it early enough. On the side note, this condition will not even compile in C#, because compiler will complain, it can not convert integer to bool implicitly. Anyway, you got the point.

One very import point I want to make is how many unit test you write? Write enough to do good code coverage. If you have a method which takes bunch of parameters and you are asserting against some values, in older version of NUnit, you would write individual methods for each assertion. With latest version of NUnit you can write the single test and decorate it with all the assertion saves lots of code duplication. It also make code maintenance easy. So please check out latest version of NUnit.

If you are lazy or do not want to do unit test yet you need to do please your boss, here is a short cut, use Microsoft PEX. In this, you point a method and PEX will create all the possible scenario test cases with all possible input parameters.

Now that I am doing more and more Silver Light + RIA, I need to do more reading on how I can test the code good. Any of you are using testing for these platform, drop me a mail.

Technorati Tags: ,