本文实例讲述了Python实现简单HTML表格解析的方法。分享给大家供大家参考。具体分析如下:
这里依赖libxml2dom,确保首先安装!导入到你的脚步并调用parse_tables() 函数。
1. source = a string containing the source code you can pass in just the table or the entire page code
2. headers = a list of ints OR a list of strings
If the headers are ints this is for tables with no header, just list the 0 based index of the rows in which you want to extract data.
If the headers are strings this is for tables with header columns (with the tags) it will pull the information from the specified columns
3. The 0 based index of the table in the source code. If there are multiple tables and the table you want to parse is the third table in the code then pass in the number 2 here
It will return a list of lists. each inner list will contain the parsed information.
具体代码如下:
#The goal of table parser is to get specific information from specific #columns in a table. #Input: source code from a typical website #Arguments: a list of headers the user wants to return #Output: A list of lists of the data in each row import libxml2dom def parse_tables(source, headers, table_index): """parse_tables(string source, list headers, table_index) headers may be a list of strings if the table has headers defined or headers may be a list of ints if no headers defined this will get data from the rows index. This method returns a list of lists """ #Determine if the headers list is strings or ints and make sure they #are all the same type j = 0 print 'Printing headers: ',headers #route to the correct function #if the header type is int if type(headers[0]) == type(1): #run no_header function return no_header(source, headers, table_index) #if the header type is string elif type(headers[0]) == type('a'): #run the header_given function return header_given(source, headers, table_index) else: #return none if the headers aren't correct return None #This function takes in the source code of the whole page a string list of #headers and the index number of the table on the page. It returns a list of #lists with the scraped information def header_given(source, headers, table_index): #initiate a list to hole the return list return_list = [] #initiate a list to hold the index numbers of the data in the rows header_index = [] #get a document object out of the source code doc = libxml2dom.parseString(source,html=1) #get the tables from the document tables = doc.getElementsByTagName('table') try: #try to get focue on the desired table main_table = tables[table_index] except: #if the table doesn't exits then return an error return ['The table index was not found'] #get a list of headers in the table table_headers = main_table.getElementsByTagName('th') #need a sentry value for the header loop loop_sentry = 0 #loop through each header looking for matches for header in table_headers: #if the header is in the desired headers list if header.textContent in headers: #add it to the header_index header_index.append(loop_sentry) #add one to the loop_sentry loop_sentry+=1 #get the rows from the table rows = main_table.getElementsByTagName('tr') #sentry value detecting if the first row is being viewed row_sentry = 0 #loop through the rows in the table, skipping the first row for row in rows: #if row_sentry is 0 this is our first row if row_sentry == 0: #make the row_sentry not 0 row_sentry = 1337 continue #get all cells from the current row cells = row.getElementsByTagName('td') #initiate a list to append into the return_list cell_list = [] #iterate through all of the header index's for i in header_index: #append the cells text content to the cell_list cell_list.append(cells[i].textContent) #append the cell_list to the return_list return_list.append(cell_list) #return the return_list return return_list #This function takes in the source code of the whole page an int list of #headers indicating the index number of the needed item and the index number #of the table on the page. It returns a list of lists with the scraped info def no_header(source, headers, table_index): #initiate a list to hold the return list return_list = [] #get a document object out of the source code doc = libxml2dom.parseString(source, html=1) #get the tables from document tables = doc.getElementsByTagName('table') try: #Try to get focus on the desired table main_table = tables[table_index] except: #if the table doesn't exits then return an error return ['The table index was not found'] #get all of the rows out of the main_table rows = main_table.getElementsByTagName('tr') #loop through each row for row in rows: #get all cells from the current row cells = row.getElementsByTagName('td') #initiate a list to append into the return_list cell_list = [] #loop through the list of desired headers for i in headers: try: #try to add text from the cell into the cell_list cell_list.append(cells[i].textContent) except: #if there is an error usually an index error just continue continue #append the data scraped into the return_list return_list.append(cell_list) #return the return list return return_list
希望本文所述对大家的Python程序设计有所帮助。
免责声明:本站资源来自互联网收集,仅供用于学习和交流,请遵循相关法律法规,本站一切资源不代表本站立场,如有侵权、后门、不妥请联系本站删除!
更新日志
- 谢金晶.2017-放心【豪记】【WAV+CUE】
- 炉石传说最强死亡骑士卡组是什么 死亡骑士最强天梯卡组推荐一览
- 炉石传说最强德鲁伊卡组是哪个 德鲁伊最强天梯卡组推荐一览
- 炉石传说最强战士卡组是哪个 战士最强天梯卡组推荐一览
- LexVandyke《HistoriadeunAmor情陷地中海》[WAV+CUE]
- TheBestPopClassics《LIGHTRHYTHMSOUND2》[WAV+CUE]
- TheLatinSoundofLexVandyke-ConciertodeAranjuez拉丁情人2012[SACD-ISO]
- 《怪物猎人世界》史低¥44,《耻辱终极版》仅需¥12
- 杉果秋促最后一天!海量3A大作击穿Steam史低
- 【果娘聊天室】国庆假期就这么水灵灵地结束啦!
- 《死或生》恋爱游戏新角色:来自大雷温柔公主的喂食
- 《寂静岭2》原制作人盛赞《寂静岭2:重制版》:新玩家能玩 我超高兴
- 艾什莉体模家今日份的COS:《鸣潮》守岸人 雪白香肩清纯而诱惑
- 微信积分商城进入方法,微信积分商城介绍
- 希诺宁进不进常驻说明,希诺宁角色分析