Why I didn’t dropped out

Ok so, first of all why at all a thought of dropping out of college? Why Do I or why I did thought about ditching the degree of which millions aspire every year and is awarded to select few. This degree which holds lots of value and anywhere I go with this degree, I would have a more than decent level of respect and admiration for. I’m doing my undergraduation in Civil Engineering at Indian Institute of Technology Bombay – Ranked no 1 in India and ~30 in World.

Well this all is one of the reason, leaving all these pride factors and convincing the stereotypical society is not easy. This is not a huge reason, if I’m asked to give weight to this reason I would say 1/100. I really don’t care about the stereotypes that exists in this society.

I don’t like Civil Engineering and after studying it for 3 years I have “NO Idea” what it is like to be a civil engineer. This kills me. I’m wasting this very precious time of my young life where I can build companies, generate employment and add a lot of value to the society instead of drawing BMDs. I love doing startups, thats what I’m made for and I’m good at it. So why not ditch it and do what I love full time. Anyways I code almost all the time of day and night.

But there is more to it. This university has provided so much, I have learned a lot of things from the people who live here and have lived here. And I believe undergraduation is not a process of becoming engineers, its more about unfolding the characters of your personality, figuring out who you want to be and how you are going to achieve it. The person you are and the ideology you follow and person you will shape up to be, its governed a lot by how you spend these four years of undergraduation. Each week I spend here, more I learn about myself and I don’t want to end this shaping up process before it is likely to be. Its a long self realizing journey, it depends on how much I can make out of it.

You make friends for life at this very beautiful place. I don’t want to leave them as of now unless our lives unfolds in a way ( which might happen after we graduate). But moving on is what life is all about, change is something that is permanent rest all is temporary, Still the rational side of my brain is not able to convince me to bail on my friends.

But the biggest reason I don’t want to drop out is I don’t feel ready for it – I just don’t feel I have “to” at this point of time. I have been asked regularly by people of all class – People with 20 years of industry experience, B-school professors, startup CEOs. (Any ways your degree is going to be of no worth to you) . It is of worth to me. Long time after I graduate it will remind me of days when I struggled, juggled between 12 hours of work accompanied with mid-terms and assignments, non stop coding sessions on nightouts and still attending that class with compulsory attendance. The time that made me who I’m now and who I’ll be in future. Learning to manage work and prioritising things. Trying to accommodate work, play, friends and family, failing at times to make it up to all, learning from mistakes and trying no to repeat them. It is what this great place has to offer me and to all the people who live here. Learning my strengths and pushing my limits in all spheres of life.

I will be working at a great organisation, no matter what; but not at the cost of ‘not being able to understand my greatest potential’ and  complement it with ‘greater set of skills’. Life  will be a learning process, It has to be. It will change in its forms from university to real job but it will be an enlightening journey.
Cheers to “Being Hungry and Being Foolish“.

Web scrapping and Automation using PhantomJS

I was recently trying to automate form filling in some sites, searching for options I came across PhantomJS. It is a great library and since I’m just too used to javascript, it turned out to be a very good choice. If you simply want to do simple form filling stuff or parse content from some specific div etc you can try CasperJS. It is a nice little layer on the top of PhantomJS, with simpler function to scrap web content or fill up forms.

So what is the exceptional thing I found about this library. All the tools so far I had used for web scraping – php-curl / python-beautifulsoup / mechanize, all them required figuring out how the form is posted, debugging / figuring out the action of your code was really tough — getting to know from console where you got stuck was really painful. And all of them required lots of code especially php-curl. Some sites would figure out that you are a bot and you get HTTP 403 error.

Now PhantomJS is very exceptional in this regard because what it does is it executes the javascript function you define like it was being executed in the JS console. ( Over the time I have become too comfortable with JS, so whenever I get a choice to use JS over other language I would choose it. So is the reason for my love for Node.js) This gives you an edge because now you just have to know how the DOM is made. Earlier there didnt used to be so much JS validation, AJAX content etc. Now forms are complex and next form fields are pulled via ahah/ajax. Unless what you do totally mimics the behavior of user it would be a tough job to scrape the web. PhantomJS comes to the resuce –

1. You dont have to worry about your user agent – it sent as the same of the computer on which your script runs and still you can modify it if required.

2. PhantomJS uses QtWebkit, so your scripts behavior is same as that of a user trying to access contents from a browser.

3. You can simple trigger events — all of them (click, change, keyup etc). This ensures that if content is fetched via AJAX – you are able to see it.

4. Debugging code is really easy. You can capture the screenshot of the browser (QtWebkit) and see upto what steps your script has accomplished its work. Still you can output stuff to console in fancy colors using CasperJS’s colorize API.

5. There are event listeners almost all of which you could imagine, i you want a custom event you have the liberty to define one. Your script has got options to wait till that click you triggered fetched that content block and not simply keep on running step by step with arbitrary timed sleep/wait.

Btw, CasperJS is widely used to write test cases for web application these days.  You should definitely give it a try. And if you are looking for an extremely powerful web scrapping and automation tool, you should head to Selenium. It is by far the most powerful tool I have come across but remember it has a huge memory heap, it will totally consume your server.

Mobile Web 101

Hey there, So it is time, you have decided to start designing/developing web app for mobile devices. Here are few basic start guides which will have you going with your mobile web application.

This will be a multi-part series, Here’s part one on differences in designing for Desktop and Mobile devices

Before you get going out there to code, I would like to tell you the difference between designing for desktop/laptops and designing for mobile devices.

There is no technical difference –  but a lot of psychological changes do occur and usage patterns do change. Mobile comes with lots of hardware possibilities. The newer HTML5 APIs are providing more and more hardware access through mobile web and helping create more and more native like experience. So what is the difference in developing for mobile platforms and PCs ? –

1. Hardware capabilities – Desktops and laptops came with the adequate amount of memory and processing capabilities, the chances of your application crashing the browser of the user was zero, so could be totally sure of that css animation not being laggy, for eg if it works on great on your computer. Mobile devices with few exceptions dont have a great hardware configurations, so your code must not be fragile,  check for memory heaps, leaks etc is a necessity, especially for Javascript intensive applications. Processing can be slow, if you are planning front-end rendering beware,  you will be facing trouble. Its not that mobile devices in general cannot process at great speeds but the problems is the multitude of devices out there, some with great quad-core processors and some with 256 Mb of RAM. You need to consider performance on an low average configuration. (Especially in a country like India where more people browse internet through phones and phones made in China. ) Also there are lots of hardware access available via HTML5 APIs which can be greatly harnessed to make user experience more pleasant. (For e.g using the location API we at housing showed directions from current location of user to the listings location, which enabled him/her to visualize on which road the listing is and decide for to visit on the move).

2. Platform + version + browser(s) variations – In the world of desktops we have Chrome, Firefox, Safari and I.E ( >7) and platforms – Windows, Mac, Linux. Through my experience whatever you design for one browser and platform it almost seamlessly works same for rest all (except for IE – which most of dont care for). When it comes to mobile devices there are platforms – Android, iOS, Windows, Bada, Symbian etc, then there are browsers chrome, android browser, firefox, opera mini, dolphin, are to name some, there are lots and lots of them and people use them to browse and in a fairly good number (#GA stats). So whats the problem? Mobile platform is very young – the implementations are buggy. So if you make something is works in one platform and browsers there are lots of chances it not working on other platform & browser. And to your surprise it may work on one version of the platform (e.g Android 4+) and not on other versions (less than Android 2.3). Coming on to browsers each on has its own implementation of interpretation of  Javascript (& support for CSS3 properties), some support some properties and some don’t, you have to have to take care of fallbacks especially in case of Javascript applications. (i got this one with History API see http://caniuse.com/#search=history )

3. Network Bandwidth/Speed and Usage Charges – This are serious concerns, you hear all the times that now mobile browsing is lightning fast etc but still in countries like India people browse over 2G/EDGE and expect lightning fast experience. Speed is something that matters the most about any web app (see Marissa Mayer on speed – http://www.youtube.com/watch?feature=player_embedded&v=WFsQvcdmLxc). Cost of ping >> Cost of content, always remember this. Instead of making each page small and dividing stylesheets and javascript files, make a multipage – single page app. Combining all files into one. Had this been the case of Desktops you could safely assume a broadband/3G dongle. Network speed is not such a greater concern as it is expected to be fast. Also you have ample screen size to show usable content on landing itself.

4. Screen Sizes – There are billions of mobile devices out there, with atleast thousands of different screen sizes, resolutions. Whatever you make must look the same on all these devices. Its not a huge challenge till you stay away from any kind of hack to put that div right there. To my surprise CSS position : fixed was such a buggy implementation on Android 2.3 (see http://caniuse.com/#feat=css-fixed) that we had to change greater part of our UX. Sticking to standards and well established CSS specs is very important and noHacks!!. Being responsive is the best way to go for a mobile applications. Now if you were to develop for a desktop, a simple % dimensions would have sufficed :P.

5. Psychology and Usage Patterns – People now-a-days use mobile devices all the times. There are generally two purposes for using a mobile web app. Either it is for passing time (social networking, gaming etc ) or for some serious need (google search, find some restaurant etc). All these second category apps need to focus a lot upon the user experience. Understanding the psychology of the user is very important. Your user came to your web app with some purpose, the quicker you can ful-fill his/her purpose beautifully and elegantly, the more happier he/she will be and more will be the chances of him/her returning. People on mobile devices hate to wait, wait for the right content to appear, navigation is already complex and people are not so keen to figure out the navigation and controls. If you are not able to serve the right taste dish in right time, the potential customer is long gone. For the first category apps, your app should be fabulous at killing time, take angry birds or temple run for eg, they serve the purpose completely. User engagement is the key, the more the user feels engaged the longer he/she stays. Keeping it simple to use and engaging is the way to go.

To conclude, there are a lots of platforms you need one app to perform and look great. This requires some serious efforts, much greater compared to what you gave on desktops. People use mobile devices on the run or for killing time. Decide the category and design UX accordingly. Some properties behave differently on different platforms and platform versions, know them and find fallbacks. I will be posting a list of such properties I feel can cause trouble to a beginner in my next blog posts. Designing a great mobile web experience is a tuff job but if you are successful in doing so, You will be the most happy Front-end Engineer out there. 🙂

Hello World!!

Ok, the same insane stereotypical title – but c’mon as a developer for every beginning there is a “Hello World”. How can I not do justice with the most simple title ever. I have been thinking about to start to blog since long but always came up with lot of excuses, lack of time, where to start on the top of list.

So what causes the change of mind all of sudden and I start this very task to pen down remarkable life learnings (mostly hacks to make your website work 😉 ). Well most of the time my problems have been solved by landing on some great (wo)men(s) blog. The internet is vast and so are the problems while making it better for the people to use. Now is the time of mobile web, people are addicted to their smartphones, some are even married to and few sleep with them. There are infinite number of devices, lot of screen sizes and resolutions, JS/CSS interpreters etc. Many developers have made lot of contribution to make desktop web easy to build and they continue their efforts for mobile web too, we all share gratitude for them. But still, the number of resources for mobile web are scarce, so its very important that we share our learnings out there and make mobile web experience better!!

So in a way via this blog I would keep posting about the new hacks I discover – keep posted.