Extract data from a calendar with Python and Beautifulsoup (under Linux Ubuntu-like)

Question

Friends,

I’d like to take data from a calendar:

http://www.purebhakti.com/component/panjika

The first step would be to make the program choose the time zone ( -3:00 Buenos Aires) and click on Submit Time Zone.

After clicking on Submit Time Zone, select the city (Rio de Janeiro) and click on Get Calendar.

Only after these steps will I have access to the calendar effectively to think about extracting the information.

I’d like to, uh, catch the event of the day:

For example, today is the 22nd, so print:

22 Apr 2017 : Ekādaśī, K, 06:09, Śatabhiṣā

+ŚUDDHA EKĀDAŚĪ VRATA: FASTING FOR Varūthinī EKADASI

I thought about using Python and beautifulsoap but I accept suggestions.

Question: How to get the program to the calendar (after selecting the time zone and city automatically)?

I couldn’t get out of:

from bs4 import BeautifulSoup
import requests

url = 'http://www.purebhakti.com/component/panjika'
header = {'user-agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) '
                        'AppleWebKit/537.36 (KHTML, like Gecko) '
                        'Chrome/51.0.2704.103 Safari/537.36'}



req = requests.get(url,headers= header)

html = req.text

soup = BeautifulSoup(html,'html.parser')

Answer 1

Try this:

import requests, time
from bs4 import BeautifulSoup as bs

url_post = 'http://www.purebhakti.com/component/panjika'
headers = {'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64)'}
payload = {'action': 2, 'timezone': 23, 'location': 'Rio de Janeiro, Brazil        043W15 22S54     -3.00'}

req = requests.post(url_post, headers=headers, data=payload)
soup = bs(req.text, 'html.parser')
eles = soup.select('tr td')
dates = (' '.join(d.select('b')[0].text.strip().split()) for d in eles if d.has_attr('class'))
events = (' '.join(d.text.split()) for d in eles if not d.has_attr('class'))
calendar = dict(zip(dates, events))

data_hoje = time.strftime("%d %b %Y", time.gmtime())
calendar[data_hoje] = calendar.setdefault(data_hoje, 'nenhum evento para hoje')
print(calendar[data_hoje])

Output from last print (today, 22 Feb 2017):

Ekādaśī, K, 05:46, Purvāṣāḍhā +ŚUDDHA EKĀDAŚĪ VRATA: FASTING FOR Vijaya EKADASI

We have to pay close attention to the HTML elements we want, in this case we want the <td>, if they have the class date is a date (dictionary key) otherwise it is the event (corresponding value)

In this case the Keys of our dictionary will be the text that is inside <b> that in turn this is inside a td that has the class attribute