This project is the machine learning related part of the entire Hebrew Court Analyzer two-semesterial project.
I have done this project alongside Immanuel Ben Hefer.
The project instructor is Jonathan Schler.
We scrape, parse, enrich verdicts from the Hebrew Supreme Court Website.
We use an ELK environment for indexing and visualizing the results.
I have used machine learning methodologies for two main parts:
There are multiple verdict categories and sub categories.
I have used tagged data to sort the verdicts by Categories.
The sorted verdicts had been tokenized by a Tfidf Vectorizer and used as an input for a Multinomial Naive Bayes model.
The classification process explained in the Classification section.
There is a variety of representations for each name in the parsed verdicts.
The normalization process tries to yield an injective function for all parsed input.
The normalization achieved by applying pre-defined deterministic steps elaborated on the Normalization section below.
- Hebrew Court Analyzer - Machine Learning - Introduction
- The verdict category classifier
- The legal personal names normalization
- Table of contents
- Data Acquisition
- Parsed json scheme
- Classification
- Normalization
- 1. Legal system related process
- 1.1. Names extraction
- 1.2. Pre-process the extracted names
- The pre-processing of each and every name consists of 5 major transformations:
- 1.3 Fixing judge names
- 1.4 Adding the newly examined name to the _legal_dictionary data member.
- 1.5 Applying normalization results to the verdict json
- 1.6 Dumping the new verdict json to the output destination
- 1.7 Dumping the new dictionary to 'legal_personal.csv' destination
- 2. Non-legal system related process
- 1. Legal system related process
- Enrichemnt Results
The verdicts scraped from the Hebrew Supreme Court Website
The parser parses those verdicts to a specific json scheme.
The parsed verdicts are ready for enrichment.
The scheme looks as follows:
{
"_index": "supreme_court_hebrew",
"_type": "rulings",
"_id": "1339-12-1",
"_score": 1,
"_source": {
"doc": {
"Doc Details": {
"מספר הליך": "",
"לפני": [
"א' רובינשטיין",
"י' עמית",
"נ' סולברג"
],
"העותר": [
],
"מידע נוסף": [
],
"המשיב": [
],
"בשם העותר": [
"עו\"ד יצחק גולדשטיין ועו\"ד ערן הלר"
],
"בשם המשיב": [
"עו\"ד מיכל דלומי"
],
"סוג מסמך": "פסק-דין",
"סיכום": "",
"תאריך": "19/03/2012",
"עמודים": 3
},
"Case Details": {}
},
"doc_as_upsert": true
}
}
The sections we mainly focus are:
The verdict id represented precisely as stored on the supreme court database - _["id"]
Judges - ["_source"]["doc"]["Doc Details"]["לפני"]
Prosecution - ["_source"]["doc"]["Doc Details"]["העותר"]
Defense - ["_source"]["doc"]["Doc Details"]["המשיב"]
Petitioner Attorneys - ["_source"]["doc"]["Doc Details"]["בשם העותר"]
Defense Attorneys - ["_source"]["doc"]["Doc Details"]["בשם המשיב"]
Verdict Summary - ["_source"]["doc"]["Doc Details"]["סיכום"]
We had a portion of tagged data which was sorted by categories and saved to a folder tree.
Those categories and sub-categories were sorted by their names and resulted on a folder tree.
I have used a pandas.DataFrame in order to store the data.
As you may have noticed, the verdicts count vary drastically. Therefore, in order to achieve good training results for each category a balancing mechasim has needed to take place.
That mechanism has sampled same quantity of verdicts from the chosen category and from the rest of the categories.
For example, Civil had 8047 verdicts - hence there will be two dataframes - one of 8047 civil verdict and the other of sampled 8047 out of 20,578 verdicts remained.
I have used train_test_split in order to split the data into 70% train and 30# test.
I have used Tfidf Vectorizer in order to transform the verdict summary - a string to an array of numbers.
The vectorizer checks how many times a word is present on a sentence, divided by the whole sentence's words.
That value is further multiplied by the logarithm of the total number of sentences divided by the sentences were the current word is present
term i within document j
w = current calculation value
The rest is plrecisely as explained above.
The transformed data from the vectorization process is the input for the model trainning.
I use a binary classification - 1 means the verdict is from the inspected category, 0 is not of that category.
I have used a Multinomial Naive Bayes Model as the model for the classification mission.
The model calculates the priors for both of the binary classes.
That means, what is the likelihood of an object to be of which class.
For instance, when 4 sentences are present, two from each class, then the prior of either class will be an half.
Then the probabilty for each word is being calulated according to that formula:
Important Note: The vocabulury is of BOTH classes.
Class in more specific manner is the verdict category.
The final model prediction result is the maximum between the product of all the words probabilities time the class prior.
The amount of words to take for each sentence, in a bag of words approach such as that, is called n_gram.
n_gram values from (1,1) to (10,10) had been set as the tfidf vectorizer input had been tested on the mnb model.
official documentation:
The results has been evalued with classification report and written to txt files.
'Civil' results for instance:
The best results had been picked.
I have written an automation that picks the optimal ngram for each and every category and trains the mnb model accordingly.
The best model and tfidf vectorizer dumped using pickle
The percedure results are two files for each category - mnb model and tfidf vectorizer.
The names we process are divided into two main categories:
- Legal system related
- Non-Legal system related
judges = verdict_json["_source"]["doc"]["Doc Details"]["לפני"]
petitioner_attorneys = verdict_json["_source"]["doc"]["Doc Details"]["בשם העותר"]
defense_attorneys = verdict_json["_source"]["doc"]["Doc Details"]["בשם המשיב"]Digits
54623
Pharenthesis content
(this is removed)
Leaked parsed statements - strings such as '2.' has leaked out from the parsing process
2.
Dash - transform
' - ' ---> '-'
Blankspaces - transform
' this ' ---> 'this'
Israeli security titles, legal titles, emergency services, etc.
Prepositions, pronouns, verbs, conjunctions etc.
Three words in length names turn into two by applying a dash between the two last names
Handles single multi-dashed names and fixing them
Looking for the pre-processed name in the judges names txt file.
That process will result on 'א חיות' turns into 'אסתר חיות'
That file has the name before normalization as key and after as the value
Those names are passing through the unwanted chars elimination process only.
The process verdict json before the process:
{
"_index": "supreme_court_hebrew",
"_type": "rulings",
"_id": "1339-12-1",
"_score": 1,
"_source": {
"doc": {
"Doc Details": {
"מספר הליך": "",
"לפני": [
"א' רובינשטיין",
"י' עמית",
"נ' סולברג"
],
"העותר": [
],
"מידע נוסף": [
],
"המשיב": [
],
"בשם העותר": [
"עו\"ד יצחק גולדשטיין ועו\"ד ערן הלר"
],
"בשם המשיב": [
"עו\"ד מיכל דלומי"
],
"סוג מסמך": "פסק-דין",
"סיכום": "",
"תאריך": "19/03/2012",
"עמודים": 3
},
"Case Details": {}
},
"doc_as_upsert": true
}
}
After:
{
"_index": "supreme_court_hebrew",
"_type": "rulings",
"_id": "1339-12-1",
"_score": 1,
"_source": {
"doc": {
"Doc Details": {
"מספר הליך": "ע\"א 1339/12",
"לפני": [
"א' רובינשטיין",
"י' עמית",
"נ' סולברג"
],
"העותר": [
"בנק מזרחי טפחות בע\"מ"
],
"מידע נוסף": [
"ערעור על פסק דינו של בית המשפט המחוזי תל אביב בת\"א 1163/00 שניתן ביום 01.01.2012 על ידי ורדה אלשיך",
"תאריך הישיבה:",
"י\"ב בשבט התשע\"ד",
"(13.01.2014)"
],
"המשיב": [
"אי.סי.אם יצרני מיזוג אויר בע\"מ (בפירוק)",
"כונס הנכסים הרשמי"
],
"בשם העותר": [
"עו\"ד יצחק גולדשטיין ועו\"ד ערן הלר"
],
"בשם המשיב": [
"עו\"ד מיכל דלומי"
],
"סוג מסמך": "פסק-דין",
"סיכום": "",
"תאריך": "19/03/2012",
"עמודים": 3,
"בשם המשיב מנורמל": [
"מיכל דלומי"
],
"העותר מנורמל": [
"בנק מזרחי טפחות בע\"מ"
],
"המשיב מנורמל": [
"אי.סי.אם יצרני מיזוג אויר בע\"מ",
"כונס הנכסים הרשמי"
],
"לפני מנורמל": [
"אליקים רובינשטיין",
"יצחק עמית",
"נעם סולברג"
],
"בשם העותר מנורמל": [
"יצחק גולדשטיין",
"ערן הלר"
]
},
"Case Details": {
"Category": "Civil"
}
},
"doc_as_upsert": true
}
}