Resolving Status Code 301 in Node.js

Today I was trying to scrape Instagram, so that I can extract some major information about the users without logging in.

Okay, I was trying to get the images of private users too.
I know it isn’t possible, but there is nothing wrong in hacking and trying to do things which are not possible.

This is the third or fourth time I am using Instagram in my life, and after logging into the account, I visited Developer’s Section just after checking some images of Russian Girls  😀

Jokes apart 😀

Now, comming to the point.
I wrote a Node.js script to collection all the information of a user but without using Instagram’s API, and when I was trying to crawl Instagram, one thing that caused problem was Status Code : 301 .

Screenshot from 2015-11-24 04:45:14

Above, the first command is executed without using following-redirects  and before executing the second command I added it to the code. Since, “/javascript” isn’t available on Instagram. So, it shown Error 404.

The last command is execute with the name of a user “/ritika” and the interesting fact is the user has a private Id on Instagram, and the user Id is extracted without using Instagram API.

I tried to figure out but my methods didn’t worked. So, I did the following steps to resolve my problem.

npm install following-redirects

After installing, I copied it from node_modules
and pasted it into node_modules   :-p

Now, all you have to do is to replace :

var http = require('http');
with
var http = require('follow-redirects').http;

and

var https = require('http');
with
var https = require('follow-redirects').https;

BOOM 😀 You can run your code now.

You can start the project at : Follow Redirects or download it directly Code

If you want to use git clone, in that case, copy the following lines :

git clone https://github.com/CodeDotJS/follow-redirects.git

P.S : When you crawl Facebook, even without using their API, for example collecting user id. Everything will work fine. But while using Instagram you’ll get this error.

Thanks 🙂 I’ll post the script when I complete it.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s