Monday 16 September 2019

GMAIL, subject checking using python.Using IMAP (Python3)


GMAIL, subject checking using python. I thought it would make an interesting post, and I will tell you why. When I was playing with the gmail notifier (about which I posted on earlier), I thought, if I can basically check if a new message has , and if it has, switch on a LED, I can pretty much make everything switch on (or off) over the internet when sending an email. I could for instance switch on an alarm system by sending an email to a gmail account with the code "344alarm2014-on" the python code would pull the email, check the subject an by some logical check switch on the device through GPIO. I can think of dozens of other applications, but let's stick with this one. Let's also stick with switching a LED on or off, to test the code. After that, whatever you want the code to invoke can be bolted on.




#source:http://bitsofpy.blogspot.com.au/2010/05/python-and-gmail-with-imap.html

import imaplib
username="someone@gmail.com"
password="somedude"

imap_server = imaplib.IMAP4_SSL("imap.gmail.com",993)
imap_server.login(username, password)

imap_server.select('INBOX')


# Count the unread emails
status, response = imap_server.status('INBOX', "(UNSEEN)")
unreadcount = int(response[0].split()[2].strip(').,]'))
print unreadcount

#Now I'm not saying this is a particularly nice way of doing #this, but if you print the response and reverse engineer 
#it you will see how I arrived with that string parsing. 
#Regex would be another option, but I try avoid that unless it is #required. 
#Now lets get a list of the identifiers for each unread message, #I'm going to call it email_ids:

# Search for all new mail
status, email_ids = imap_server.search(None, '(UNSEEN)')
print (email_ids)

def get_emails(email_ids):
    data = []
    for e_id in email_ids:
        _, response = imap_server.fetch(e_id, '(UID BODY[TEXT])')
        data.append(response[0][1])
    return data

def get_subjects(email_ids):
    subjects = []
    for e_id in email_ids:
        _, response = imap_server.fetch(e_id, '(body[header.fields (subject)])')
        subjects.append( response[0][1][9:] )
    return subjects

#And I often search for emails from someone in particular, I can #do that easily from Python as well:

def emails_from(name):
    '''Search for all mail from name'''
    status, response = imap_server.search(None, '(FROM"%s")'%name)
    email_ids = [e_id for e_id in response[0].split()]
    print ('Number of emails from %s: %i. IDs: %s' % (name, len(email_ids), email_ids))
    return email_ids



something worth checking:
#!/usr/bin/env python

import subprocess, feedparser

DEBUG = 1

USERNAME = "myusername"     # just the part before the @ sign, add yours here
PASSWORD = "mypassword"     

NEWMAIL_OFFSET = 0        # my unread messages goes to zero, yours might not
MAIL_CHECK_FREQ = 60      # check mail every 60 seconds


while True:

        newmails = int(feedparser.parse("https://" + USERNAME + ":" + PASSWORD +"@mail.google.com/gmail/feed/atom/sunset")["feed"]["fullcount"])

        if newmails > NEWMAIL_OFFSET:
                subprocess.call(["python relay-test.py",],shell=True)
        else:
                pass

        time.sleep(MAIL_CHECK_FREQ)


background info:


http://bruno.im/2010/jul/29/europython-talk-python-and-imap-protocol/

https://pythonadventures.wordpress.com/tag/gmail/


gmail python library:

https://github.com/charlierguo/gmail

Python 3, using Split (and Tuples)


Split [generic definition and capabilities]




The simplest way to use split is by defining a line of something, then split the words up, implicitly, using a space as delimiter:


>>> line="just some typing as an illustration"
>>> line.split()
['just', 'some', 'typing', 'as', 'an', 'illustration']

using str.split(" ") would have the same result; the sentence is turned into a list.


line="just some sentence to illustrate something"
words_list=line.split()
print(words_list)      

Or, for example print(word-list[2][1])  to print from word 2  (typing) the letter at position 1 (y) (remember couting starts at 0)





So I have created a csv file, using xls, see below. The good things about csv file is that it is built up by continuous rows, delimited by a comma. It is therefore no more than logical to use the split command to separate the data again.



summing up the company name, number of shares and price.




# File containing lines of the form "name,shares,price"
filename = "shares.csv"
portfolio = []
for line in open(filename):
    fields = line.split(",") # Split each line into a list using "," as the delimiter.
    name = fields[0] # Extract and convert individual fields
    shares = int(fields[1])
    price = float(fields[2])
    stock = (name,shares,price) # Create a tuple (name, shares, price)
    portfolio.append(stock) # Append to list of records



Call the tuple for instance:

>>>portfolio[0]
('Philips', 100, 100.5)



Python Google/Gmail API

In order to have something tap into your Google API, you need to set up OAuth. Now before I continue, I just to be able to tap into my Gmail account, using the python imaplib, I am no longer able to access the account through python anymore.  Google claims that the Gmail API is no substitution for imap, but i am not sure. So anyone who has some insights on imaplib and gmail and if it still works without OAuth, please leave a comment below.

Yes, so out of necessity I started looking into OAuth, python and Gmail, and it actually does not look that bad.


What is OAuth?

OAuth is essentially a way, to avoid having to write all sorts of authentication code into your application (in our case the pythin script).





I dont want to dwell on Oauth to much, as this post is more of an instruction on how to access the google GMAIL API. 



Instructions

1-Open google account or use existing one and go to Developers section.

2-https://developers.google.com/gmail/api/

3-Go to the Guides TAB (see below), which pretty much tells you everything I was gonna tell you but without the spelling mistakes.




4- run the pip install:


pip install --upgrade google-api-python-client google-auth-httplib2 google-auth-oauthlib

pip install --upgrade google-api-python-client google-auth-httplib2 google-auth-oauthlib
pip install --upgrade google-api-python-client google-auth-httplib2 google-auth-oauthlib


5-download the client configuration file, which contains the client ID and token. Put this in same directory as the python file you are about to add.

6. Create python script:


from __future__ import print_functionimport pickleimport os.pathfrom googleapiclient.discovery import buildfrom google_auth_oauthlib.flow import InstalledAppFlow
from google.auth.transport.requests import Request
# If modifying these scopes, delete the file token.pickle.
SCOPES = ['https://www.googleapis.com/auth/gmail.readonly']
def main():
    """Shows basic usage of the Gmail API.
    Lists the user's Gmail labels.
    """
    creds = None
    # The file token.pickle stores the user's access and refresh tokens, and is
    # created automatically when the authorization flow completes for the first
    # time.
    if os.path.exists('token.pickle'):
        with open('token.pickle', 'rb') as token:
            creds = pickle.load(token)
    # If there are no (valid) credentials available, let the user log in.
    if not creds or not creds.valid:
        if creds and creds.expired and creds.refresh_token:
            creds.refresh(Request())
        else:
            flow = InstalledAppFlow.from_client_secrets_file(
                'credentials.json', SCOPES)
            creds = flow.run_local_server(port=0)
        # Save the credentials for the next run
        with open('token.pickle', 'wb') as token:
            pickle.dump(creds, token)

    service = build('gmail', 'v1', credentials=creds)

    # Call the Gmail API
    results = service.users().labels().list(userId='me').execute()
    labels = results.get('labels', [])

    if not labels:
        print('No labels found.')
    else:
        print('Labels:')
        for label in labels:
            print(label['name'])
if __name__ == '__main__':
    main()
from __future__ import print_functionimport pickleimport os.pathfrom googleapiclient.discovery import buildfrom google_auth_oauthlib.flow import InstalledAppFlow from google.auth.transport.requests import Request # If modifying these scopes, delete the file token.pickle. SCOPES = ['https://www.googleapis.com/auth/gmail.readonly'] def main():     """Shows basic usage of the Gmail API.     Lists the user's Gmail labels.     """     creds = None     # The file token.pickle stores the user's access and refresh tokens, and is     # created automatically when the authorization flow completes for the first     # time.     if os.path.exists('token.pickle'):         with open('token.pickle', 'rb') as token:             creds = pickle.load(token)     # If there are no (valid) credentials available, let the user log in.     if not creds or not creds.valid:         if creds and creds.expired and creds.refresh_token:             creds.refresh(Request())         else:             flow = InstalledAppFlow.from_client_secrets_file(                 'credentials.json', SCOPES)             creds = flow.run_local_server(port=0)         # Save the credentials for the next run         with open('token.pickle', 'wb') as token:             pickle.dump(creds, token)     service = build('gmail', 'v1', credentials=creds)     # Call the Gmail API     results = service.users().labels().list(userId='me').execute()     labels = results.get('labels', [])     if not labels:         print('No labels found.')     else:         print('Labels:')         for label in labels:             print(label['name']) if __name__ == '__main__':     main()
from __future__ import print_functionimport pickleimport os.pathfrom googleapiclient.discovery import buildfrom google_auth_oauthlib.flow import InstalledAppFlow from google.auth.transport.requests import Request # If modifying these scopes, delete the file token.pickle. SCOPES = ['https://www.googleapis.com/auth/gmail.readonly'] def main():     """Shows basic usage of the Gmail API.     Lists the user's Gmail labels.     """     creds = None     # The file token.pickle stores the user's access and refresh tokens, and is     # created automatically when the authorization flow completes for the first     # time.     if os.path.exists('token.pickle'):         with open('token.pickle', 'rb') as token:             creds = pickle.load(token)     # If there are no (valid) credentials available, let the user log in.     if not creds or not creds.valid:         if creds and creds.expired and creds.refresh_token:             creds.refresh(Request())         else:             flow = InstalledAppFlow.from_client_secrets_file(                 'credentials.json', SCOPES)             creds = flow.run_local_server(port=0)         # Save the credentials for the next run         with open('token.pickle', 'wb') as token:             pickle.dump(creds, token)     service = build('gmail', 'v1', credentials=creds)     # Call the Gmail API     results = service.users().labels().list(userId='me').execute()     labels = results.get('labels', [])     if not labels:         print('No labels found.')     else:         print('Labels:')         for label in labels:             print(label['name']) if __name__ == '__main__':     main()

Now when you run the script, it will generate an authorization link:






This link is basically a confirmation authorization, i.e. grant authorization to the application to access the API. I will prompt you to grant access to the python "application" to be able to access the gmail account.



This is a once off, after this, tokens will be exchanged.

For access to your API and to down oad the client json file:


https://console.developers.google.com


This particular python code, prints all the labels in the gmail account.

Monday 28 August 2017

PYTHON setting environment variables and PATHS



Setting windows environment variables for python:

Typically python will install itself in:

C:\users\<user-name>\appdata\local\programs\python<version>\

Win 7: CONTROL PANEL  >  SYSTEM  >  ADVANCED SYSTEM SETTINGS  > 
ENVIRONMENT VARIABLES


Win 10; This PC > Properties > Advanced System Settings > Environment Variables

then add a new system variable.

For example PYTHONPATH=C:\pycode\utilities

(c:\pycode\utilities could really be anything you stick your code and libraries in, but because python install on appData, you can put in

C:\Users\<user-name>\AppData\Local\Programs\Python\Python38-32\   )






next you will set the Path system variable:

go to system variable and EDIT


Now add the PATH you set in the first step, using % at start and end:

in the 'edit environment variable' add NEW, i called my previous environment variable PYTHONPATH, so that will go in:





now you can verify, by opening a command prompt and, typing 'python:


which should now allow you to run python from any directory without having to go to the directory where python is installed
In python IDLE this directory should now show up in your path browse.




Monday 23 December 2013

Python, Gmail notification


In this post I will start exploring Python. I will obviously start off with a simple example (which I sort of borrowed)

Before using python on a Raspberry pi, python will need to be installed:


sudo apt-get install python python-dev python-pip
sudo easy_install -U distribute  

More than likely you will use python in conjunction with your Pi's GPIO, in which case you will need the rpi.gpio python module. The command below will alsoload the feedparser which alows RSS and atom feeds to be parsed into the python script.


sudo pip install feedparser RPi.GPIO
Downloading/unpacking feedparser
  Downloading feedparser-5.1.3.tar.bz2 (202Kb): 202Kb downloaded

Please note, that the script below is based on python 2.7. I found this a scripta good start to play around with python and GPIO at the same time. What thescript does, is check for the email account for new emails and if a new unread item is in the inbox, it will turn on the LED on pin 12.
   





First of all, the RPi.GPIO and feedparser modules are loaded





import RPi.GPIO as GPIO, feedparser    <load GPIO and feedparser module
USERNAME="username@gmail.com"
PASSWORD="password"
GPIO_PIN=12     <set the string
GPIO.setmode(GPIO.BOARD)
GPIO.setup(GPIO_PIN, GPIO.OUT)
newmails = int(feedparser.parse("https://" + USERNAME + ":" + PASSWORD + "@mail.google.com/gmail/feed/atom")["feed"]["fullcount"])
set the integer "newmails"
if newmails > 0: 
 GPIO.output(GPIO_PIN, True) <LED on pin 12 on
else: 
 GPIO.output(GPIO_PIN, False)<LED on pin 12 off


















source:  
http://mitchtech.net/raspberry-pi-physical-gmail-notifier/

I have found that when running the script and running it's python debug, I initially got an error message suggesting to run the script as root. In order to do this, I start up IDLE (Or IDLE 3 for Python 3.3 users) from a terminal or root terminal and just enter sudo idle/idle3 or idle/idle3 respectively.

oh yeah, almost forgot, here is the bread board pin out.



Merry Xmas!













Friday 29 November 2013

Raspberry Pi Windows Shared folders, Samba/CIFS client,

Another post that I has lying around in scratch book status for ages, so time to finalise it. This post is based on version 3.6.11+, most of my Raspi is unchanged an very stock standard, so I would assume mounting your drive should work using my instructions below.

SAMBA was originally SMB Server - but the name had to be changed due to SMB Server being an actual product. SMB was the predecessor to CIFS. SMB "server message block" and CIFS "common internet file system" are protocols. Samba implements CIFS network protocol. This is what allows Samba to communicate with (newer) MS Windows systems. Typically you will see it referred to SMB/CIFS. However, CIFS is the extension of the SMB protocol so if someone is sharing out SMB via Samba to a legacy system still using NetBIOS it will typically connect to the Samba server via port 137, 138, 139 and CIFS is strictly port 445 (You will see this is you spin up wireshark on a test machine). 

First of all, verify if the cifs module is installed

pi@raspberrypi ~ $ dpkg-query -S cifs-utils
cifs-utils: /usr/share/doc/cifs-utils
cifs-utils: /usr/share/doc/cifs-utils/changelog.gz
cifs-utils: /usr/share/doc/cifs-utils/NEWS.Debian.gz
cifs-utils: /usr/share/doc/cifs-utils/README
cifs-utils: /usr/share/doc/cifs-utils/changelog.Debian.gz
cifs-utils: /usr/share/doc/cifs-utils/copyright


To verify if the samba software has been installed,  by issuing:

pi@raspberrypi ~ $ dpkg -l | grep samba-common
ii  samba-common      2:3.6.6-6+deb7u1      all common files used by both the Samba server and client

Or issue   dpkg -p samba-common  to check its dependencies. 


First all, you will create a mount point, this is nothing more then a directory that will contain the contents of the shared drive or folder once it is mounted.

Do this by creating a directory, this cold be in your home folder or where ever you want, I have chose to use /mnt/ to keep things consistent:



mkdir /mnt/TPLINK_NAS1

Now mount the drive to the mount point for example:


mount -t cifs -o sec=none //10.19.19.102/volume1/cams/front1 /mnt/TPLINK_NAS1


As you will notice I have used the sec=none switch, this because there is an issue with Raspian, as of version 3.10.19, that causes problems when using username and password. Or so is the consensus on the raspi.org forum:

Source:
http://www.raspberrypi.org/phpBB3/viewtopic.php?f=66&t=60699&p=459637#p459637

What this does is actually explicitly link the /volume1/cams/front1 directory to mnt/TPLINK_NAS, which means everythging that gets dumped into the mount point, essentially gets dumped onto the share. Very much like mapping shares in Windows, but better.


Permanent shares at start up. Well of course the above mount command lasts till reboot, or otherwise explicitly unmounted. In order to mound a share to a mount point at start up. the place to do this is:


/etc/fstab

I have edited fstab similar to the mount command, as per below

proc            /proc           proc    defaults          0       0
/dev/mmcblk0p5  /boot           vfat    defaults          0       2
/dev/mmcblk0p6  /               ext4    defaults,noatime  0       1
# a swapfile is not a swap partition, so no using swapon|off from here on, use  dphys-swapfile swap[on|off]  for that
//10.19.19.102/volume1/cams/front1 /mnt/TPLINK_NAS1 cifs guest,users,uid=1000,auto,_netdev 0 0

To verify If your mount is functioning, issue:

cat /proc/mounts


Goodluck

Sources:

Full book on Samba:

http://www.samba.org/samba/docs/using_samba/ch00.html



https://wiki.samba.org/index.php/Mounting_samba_shares_from_a_unix_client



http://rasspberrypi.wordpress.com/2012/09/04/mounting-and-automounting-windows-shares-on-raspberry-pi/