Using NLTK to analyse vocabulary in political manifestos

The technical process:

1. download the manifesto pdf files
2. convert the pdf files to text files
3. use NLTK to remove stopwords and get the common words used in each manifesto

​This required two scripts, the first converted the pdf files to text files, I used the script here. The second script is on my code blog.

The results:
DUP
[('northern', 147), ('ireland', 102), ('n', 99), ('dup', 88), ('new', 40), 
('westminster', 29), ('uk', 26), ('support', 25), ('public', 25), ('national', 21), 
('people', 18), ('united', 18), ('deal', 17), ('needs', 17), ('manifesto', 16), 
('trade', 16), ('must', 14), ('also', 14), ('best', 14), ('wminster', 13), ('eu', 13), 
('across', 13), ('irelands', 13), ('economic', 13), ('identity', 13), ('military', 12), 
('would', 12), ('better', 12), ('positive', 12), ('believes', 12)]


UUP
[('northern', 51), ('ireland', 33), ('ulster', 23), ('unionist', 22), ('best', 18), 
('united', 16), ('party', 16), ('brexit', 13), ('westminster', 13), ('mps', 12), 
('local', 12), ('union', 12), ('work', 11), ('manifesto', 11), ('one', 11), ('remain', 9), 
('great', 8), ('need', 8), ('many', 8), ('yet', 8), ('would', 8), ('public', 7), 
('south', 7), ('tom', 7), ('executive', 7), ('kingdom', 7), ('people', 7), ('election', 7), 
('danny', 6), ('better', 6)]

Alliance
[('northern', 170), ('alliance', 110), ('ireland', 103), ('uk', 75), 
('support', 65), ('ensure', 51), ('would', 43), ('change', 40), ('public', 37), 
('economic', 35), ('also', 32), ('westminster', 32), ('work', 30), ('government', 30), 
('european', 29), ('direction', 27), ('need', 27), ('deal', 26), ('across', 26), 
('executive', 26), ('international', 25), ('political', 25), ('brexit', 24), 
('must', 24), ('eu', 24), ('significant', 23), 
('continue', 23), ('tax', 23), ('human', 22), ('welfare', 22)]

Sinn Fein
[('brexit', 44), ('eu', 35), ('sinn', 32), ('rights', 30), ('tory', 25), ('health', 23), 
('north', 23), ('irish', 19), ('access', 18), ('european', 17), ('vote', 16), 
('within', 16), ('party', 15), ('special', 14), ('services', 13), ('ireland', 13), 
('status', 13), ('election', 13), ('cuts', 12), ('funding', 12), ('designated', 12), 
('dup', 12), ('new', 11), ('unity', 10), ('trade', 9), ('priorities', 9), ('people', 9), 
('british', 8), ('good', 8), ('friday', 8)]


TUV
[('tuv', 97), ('northern', 95), ('ireland', 81), ('politics', 42), ('would', 41), 
('principled', 41), ('talking', 41), ('straight', 41), ('stormont', 35), ('sinn', 34), 
('fein', 30), ('people', 27), ('need', 27), ('dup', 22), ('government', 22), ('public', 20), 
('believes', 19), ('first', 19), ('uk', 19), ('must', 18), ('irish', 17), ('health', 16), 
('brexit', 15), ('eu', 15), ('one', 15), ('money', 15), ('assembly', 15), ('could', 14), 
('language', 14), ('mental', 14)]


SDLP
[('sdlp', 262), ('northern', 154), ('ireland', 104), ('new', 97), ('ensure', 85), 
('must', 58), ('support', 58), ('people', 55), ('believes', 51), ('also', 45), 
('investment', 39), ('public', 39), ('housing', 39), ('education', 39), ('better', 37), 
('government', 37), ('local', 35), ('development', 35), ('services', 32), ('strategy', 32), 
('economy', 31), ('across', 31), ('health', 30), ('areas', 30), ('community', 30), 
('sector', 29), ('economic', 28), ('work', 27), ('social', 27), ('future', 26)]
​Note the SDLP manifesto for the Westminster election can not be scanned using the pdf to text script so I scanned their Stormont manifesto from March 2017.

Analysis:
All of the parties, except Sinn Fein, mention Northern Ireland a lot. Sinn Fein refuse to use the name Northern Ireland - to admit the existence of Northern Ireland would be to recognise that the Unionists have some validity in their views, they refuse to do that. Northern Irish politics can be juvenile. Sinn Fein have put Brexit and the EU in the centre of their message. The two main Unionist parties also talk about Brexit and the EU but not as prominently. The TUV prefer to talk about Sinn Fein, the DUP and Stormont.
The opinions of the five main parties to Brexit are:
DUP: make sure Northern Ireland gets the best Brexit deal
UUP: that all the constituent regions of our country (the UK) remain strong and united
Alliance: Opposition to a hard Brexit
SDLP: The Good Friday Agreement clearly makes Northern Ireland distinct from the rest of the UK
Sinn Fein: Brexit  and  Tory  cuts  demonstrate the undemocratic nature of partition

The UUP position reflects some concern within elements of the Unionist community that Brexit could be the end of the UK. The nationalists prefer to focus on the remain vote in Northern Ireland and want special status for Northern Ireland. The DUP will reject this idea but there is some evidence that Brexit has weakened the UK and increased talk of a United Ireland.