Parse HTML table data to JSON and save to text file in Python 2.7 -


i'm trying extract data on crime rate across states webpage, link web page http://www.disastercenter.com/crime/uscrime.htm

i able text file. response in json format. how can in python.

here code:

import urllib         import re       bs4 import beautifulsoup     link = "http://www.disastercenter.com/crime/uscrime.htm"     f = urllib.urlopen(link)     myfile = f.read()     soup = beautifulsoup(myfile)     soup1=soup.find('table', width="100%")     soup3=str(soup1)     result = re.sub("<.*?>", "", soup3)     print(result)     output=open("output.txt","w")     output.write(result)     output.close()

the following code data 2 tables , output of json formatted string.


working example (python 2.7.9):

from lxml import html import requests import re regular_expression import json  page = requests.get("http://www.disastercenter.com/crime/uscrime.htm") tree = html.fromstring(page.text)  tables = [tree.xpath('//table/tbody/tr[2]/td/center/center/font/table/tbody'),           tree.xpath('//table/tbody/tr[5]/td/center/center/font/table/tbody')]  tabs = []  table in tables:     tab = []     row in table:         col in row:             var = col.text_content()             var = var.strip().replace(" ", "")             var = var.split('\n')             if regular_expression.match('^\d{4}$', var[0].strip()):                 tab_row = {}                 tab_row["year"] = var[0].strip()                 tab_row["population"] = var[1].strip()                 tab_row["total"] = var[2].strip()                 tab_row["violent"] = var[3].strip()                 tab_row["property"] = var[4].strip()                 tab_row["murder"] = var[5].strip()                 tab_row["forcible_rape"] = var[6].strip()                 tab_row["robbery"] = var[7].strip()                 tab_row["aggravated_assault"] = var[8].strip()                 tab_row["burglary"] = var[9].strip()                 tab_row["larceny_theft"] = var[10].strip()                 tab_row["vehicle_theft"] = var[11].strip()                 tab.append(tab_row)     tabs.append(tab)  json_data = json.dumps(tabs)  output = open("output.txt", "w") output.write(json_data) output.close()

Comments

Post a Comment

Popular posts from this blog

c++ - Difference between pre and post decrement in recursive function argument -

i have following sample code used pre-decrement void function(int c) { if(c == 0) { return; } else { cout << "dp" << c << endl; function(--c); cout << c << endl; return; } } this function give output input 4 : dp3 dp2 dp1 dp0 0 1 2 3 but when use post decrement void function(int c) { if(c == 0) { return; } else { cout << "dp" << c << endl; function(c--); cout << c << endl; return; } } output goes in infinite loop dp4 can explain me in detail why happens ? this happens because function(c--) called same value of c , when finishes c decremented. recursively called, hence called same value no chance of return , hit stack overflow error. assume this, int x = 1, y = 0; y = x++ result y == 1 , x == 2. if y = ++x , bot x , y 2.
Read more

php - Nothing but 'run(); ' when browsing to my local project, how do I fix this? -

i added new project. made mistake in projectname.conf file in sites-available apache2, led me reinstalling apache2. after reinstalling , re-enabling projects on every page of every project .. <body> run(); </body> .. i've run composer update, restarted service, restarted pc, checked config files, tried disabling new project. there nothing in error log, , access.log: 127.0.0.1 - - [15/may/2015:01:40:47 +0200] "get / http/1.1" 304 181 "-" "mozilla/5.0 (x11; ubuntu; linux x86_64; rv:38.0) gecko/20100101 firefox/38.0" maybe i'm forgetting apache2 reinstall? don't think did different last time installed it. looks haven't setup php correctly. take @ this great guide , maybe you're forgetting link php config apache2.
Read more

php - How can I echo out this array? -

i don't know how go grabbing data in variable , put foreach loop. class tree{ private $info = array( array('name'=>'jim','state'=>'nu','company'=>'nu','phone'=>array('cell'=>'5615111111','office'=>'5611111111'),'email'=>array('primary'=>'exs@example.com','ex@there.com')), array('name'=>'joe smith','city'=>'phoenix','phone'=>'4805551111','email'=>'jsmith@some_email.com'), array('name'=>'john doe','city'=>'chandler','company'=>'doe co.','email'=>array('jdoe@gmail.com','personal'=>'email@email.com'),'phone'=>'6025550002') ); } if you're inside of class, can access private variable using $this->variablename . example, i...
Read more