Actuarial Outpost
 
Go Back   Actuarial Outpost > Actuarial Discussion Forum > Software & Technology
FlashChat Actuarial Discussion Preliminary Exams CAS/SOA Exams Cyberchat Around the World Suggestions

Search Actuarial Jobs by State @ DWSimpson.com:
AL AK AR AZ CA CO CT DE FL GA HI ID IL IN IA KS KY LA
ME MD MA MI MN MS MO MT NE NH NJ NM NY NV NC ND
OH OK OR PA RI SC SD TN TX UT VT VA WA WV WI WY

Reply
 
Thread Tools Search this Thread Display Modes
  #1  
Old 06-26-2020, 10:58 AM
binyamin binyamin is offline
Member
Non-Actuary
 
Join Date: Jul 2013
Posts: 253
Default Fun with python

Has anyone done web scraping for fun using python? Like simple legal stuff, going to a website give your credentials, search, and download. Which package did you use? Has anyone come across good example codes using "Beautiful Soup"? I googled, but I want to know any of the readers here have used it.
Reply With Quote
  #2  
Old Yesterday, 08:47 PM
dmbymdt's Avatar
dmbymdt dmbymdt is offline
Member
SOA AAA
 
Join Date: Sep 2010
Location: Hartford, CT
College: Cold and Snowy University
Favorite beer: Double IPA's
Posts: 235
Default

Quote:
Originally Posted by binyamin View Post
Has anyone done web scraping for fun using python? Like simple legal stuff, going to a website give your credentials, search, and download. Which package did you use? Has anyone come across good example codes using "Beautiful Soup"? I googled, but I want to know any of the readers here have used it.
I used beautifulsoup for web scraping james allen website. Since doing this james allen actually changed their website design and didn't have all of the same information. It's a good scraper for python. https://pypi.org/project/beautifulsoup4/

A few others I used

import pandas as pd
import urllib3
import certifi
import os
import time
from bs4 import BeautifulSoup
from collections import namedtuple
from json import loads
import requests
import json

Many sites have api's and it is suggested to use this structured where possible.
__________________
ASA CERA LFV-US LP

Reply With Quote
Reply

Thread Tools Search this Thread
Search this Thread:

Advanced Search
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

BB code is On
Smilies are On
[IMG] code is On
HTML code is Off


All times are GMT -4. The time now is 08:12 AM.


Powered by vBulletin®
Copyright ©2000 - 2020, Jelsoft Enterprises Ltd.
*PLEASE NOTE: Posts are not checked for accuracy, and do not
represent the views of the Actuarial Outpost or its sponsors.
Page generated in 0.19264 seconds with 9 queries