1.py

# -*- coding: utf-8 -*-
"""Job 15 Career skill Dataset.ipynb

Automatically generated by Colab.

Original file is located at
    https://colab.research.google.com/drive/1RTQ3VjxFUAcboaf3ls_bkNqKaeWUJVDA

The task is to build a career skill dataset that organizes information about different careers and the skills required for them. The dataset links career roles with domains, skill levels, difficulty, thresholds, and learning resources. This structured dataset can be used for career guidance, skill assessment, and machine learning applications such as career recommendation systems.

From the dataset these are the columns in it:
	•	Career
	•	Domain
	•	Skill/Sub-skill
	•	Level Required
	•	Difficulty
	•	Threshold (0–100)
	•	Weight
	•	Question Category
	•	Sample Quiz Prompt
	•	Resource 1
	•	Resource 2
	•	Resource 3
	•	Tags

Target Variable (Dependent Variable)

The target variable is the column the model is trying to predict.

For this dataset, the most logical target variable is:

Career

This means the machine learning model will learn from the skills and features and try to predict which career path they belong to. Example careers in the dataset:
	•	Data Engineer
	•	AI Engineer
	•	Machine Learning Engineer
	•	Cloud Engineer

Feature Variables (Independent Variables)

The feature variables are the inputs used by the model to make predictions.

Possible features in this dataset:
	•	Domain
	•	Skill/Sub-skill
	•	Level Required
	•	Difficulty
	•	Threshold (0–100)
	•	Weight
	•	Question Category
	•	Tags

Why the task matters?  

The task matters because building a career skill dataset provides structured information about careers and the skills required for them. This dataset can be used for career guidance, skill gap analysis, and machine learning applications such as career recommendation systems. It also helps learners understand which skills they need to develop for specific career paths.

Dataset Overview Report

1. Number of Rows and Columns

The dataset contains:
	•	1000 rows
	•	13 columns

Each row represents a career skill entry, describing the skills, difficulty level, resources, and other attributes related to a specific career.


2. Column Names and Meanings

Career:
The job role associated with the skill (e.g., Data Scientist, AI Engineer).

Domain:
The broader field or industry of the career (e.g., Data Science, Cloud Computing).

Skill/Sub-skill:
The specific skill required for that career.

Level Required:
The proficiency level needed (Beginner, Intermediate, Advanced).

Difficulty:
Indicates how difficult the skill is to learn.

Threshold (0–100):
A score indicating the minimum competency level required.

Weight:
Importance of the skill for that career.

Question Category:
The type of assessment question related to the skill.

Sample Quiz Prompt:
Example question used to test the skill.

Resource 1:
First learning resource for the skill.

Resource 2:
Second learning resource for the skill.

Resource 3:
Third learning resource for the skill.

Tags:
Keywords describing the skill or domain.

3. Data Types

The dataset contains both categorical and numerical variables.

Categorical Columns
	•	Career
	•	Domain
	•	Skill/Sub-skill
	•	Level Required
	•	Difficulty
	•	Question Category
	•	Sample Quiz Prompt
	•	Resource 1
	•	Resource 2
	•	Resource 3
	•	Tags

These are stored as object (string) data types.

Numeric Columns
	•	Threshold (0–100) → Integer
	•	Weight → Float

These represent numeric measures of skill importance and required competency.

4. Number of Unique Values per Column

Career - 15

Domain - 10

Skill/Sub-skill - 97

Level Required - 3

Difficulty - 3

Threshold (0–100) - 3

Weight - 2

Question Category - 10

Sample Quiz Prompt - 22

Resource 1 - 9

Resource 2 - 9

Resource 3 - 9

Tags - 15


This shows that several columns contain limited variation, especially the numeric columns.

5. Missing Values

The dataset contains:

No missing values.

Every column has complete data for all 1000 rows.

6. Duplicate Rows

The dataset contains:

300 duplicate rows

This indicates that several entries are repeated.

Duplicates may occur because:
	•	multiple careers share similar skill structures
	•	the dataset may have been generated using a template.

7. Nature of the Dataset (Synthetic or Real)

Based on the structure of the dataset, it is likely synthetic or highly templated.

Evidence includes:
	•	No missing values across all columns
	•	Many duplicate rows (300 duplicates)
	•	Several categorical columns with limited unique values
	•	Numeric columns (Threshold and Weight) show very limited variability
	•	Repeated resource links and quiz prompts

These characteristics suggest that the dataset was programmatically generated or built using structured templates rather than collected from raw real-world data.

Import and load dataset
"""

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns

from sklearn.model_selection import train_test_split
from sklearn.preprocessing import LabelEncoder
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score, confusion_matrix, classification_report

df = pd.read_excel("New jobready_career_skill_dataset_1000_rows_v2.xlsx")
df.head()

"""Dataset Overview"""

df.shape

"""Check columns name"""

df.columns

"""Check data types"""

df.info()

"""Check missing values"""

df.isnull().sum()

"""Insight:

The dataset contains no missing values, meaning data cleaning will be minimal.

Check Duplicates
"""

df.duplicated().sum()

"""Remove Duplicates"""

df = df.drop_duplicates()

"""Insight:

The dataset contained 300 duplicate rows, which were removed to improve data quality.

Univariate Analysis
"""

plt.figure()
sns.countplot(x='Career', data=df)
plt.xticks(rotation=90)
plt.title("Career Distribution")
plt.show()

"""Insight:

Careers are fairly evenly distributed, so class imbalance is not a major issue.

Domain Distribution
"""

plt.figure()
sns.countplot(x='Domain', data=df)
plt.xticks(rotation=90)
plt.title("Domain Distribution")
plt.show()

"""Insight:

Most domains appear with similar frequencies, suggesting the dataset was structured or templated.

Level Required
"""

sns.countplot(x='Level Required', data=df)
plt.title("Level Required Distribution")
plt.show()

"""Insight:

Intermediate level dominates, indicating most skills target mid-level professionals.

Difficulty
"""

sns.countplot(x='Difficulty', data=df)
plt.title("Difficulty Distribution")
plt.show()

"""Insight:

Most skills are labeled Medium difficulty, aligning with intermediate skill levels.

Numeric Variable Analysis

Threshold
"""

plt.figure()
df['Threshold (0-100)'].hist()
plt.title("Threshold Distribution")
plt.show()

"""Insight:

Threshold values have limited variability, suggesting predefined competency levels.

Weight
"""

df['Weight'].hist()
plt.title("Weight Distribution")
plt.show()

"""Insight:

The weight column contains very little variation, meaning it may not strongly influence predictions.

Bivariate Analysis

Career vs Domain
"""

pd.crosstab(df['Career'], df['Domain'])

"""Insight:

Careers share several domains, indicating cross-domain skill requirements.

Career vs Difficulty
"""

pd.crosstab(df['Career'], df['Difficulty'])

"""Insight:

Medium difficulty dominates across careers.

Career vs Level Required
"""

pd.crosstab(df['Career'], df['Level Required'])

"""Insight:

Most careers emphasize intermediate skill levels.

Feature Selection

Remove columns that are not useful for machine learning.
"""

df_model = df.drop(columns=[
    'Sample Quiz Prompt',
    'Resource 1',
    'Resource 2',
    'Resource 3'
])

"""Insight:

These columns contain learning resources and text templates rather than predictive information.

Encode Categorical Variables

Machine learning models require numerical values.
"""

le = LabelEncoder()

for col in df_model.select_dtypes(include='object').columns:
    df_model[col] = le.fit_transform(df_model[col])

"""Define Features and Target

Target variable:
"""

y = df_model['Career']

"""Feature Variables:"""

x = df_model.drop('Career', axis=1)

"""Train-Test Split"""

x_train, x_test, y_train, y_test = train_test_split(
    x, y, test_size=0.2, random_state=42
)

"""This means:
	•	80% training data
	•	20% testing data

Train Machine Learning Model

Using Logistic Regression.
"""

model = LogisticRegression(max_iter=200)

model.fit(x_train, y_train)

"""Make Predictions"""

y_pred = model.predict(x_test)

"""Model Evaluation

Accuracy Score
"""

accuracy = accuracy_score(y_test, y_pred)

print("Accuracy:", accuracy)

"""Insight:

The model correctly predicts careers around 52.5% of the time.

Confusion Matrix
"""

cm = confusion_matrix(y_test, y_pred)

plt.figure()
sns.heatmap(cm, annot=True)
plt.title("Confusion Matrix")
plt.show()

"""Insight:

The confusion matrix shows how often careers are correctly classified or misclassified.

Classification Report
"""

print(classification_report(y_test, y_pred))

"""This shows:
	•	Precision
	•	Recall
	•	F1-score

Insight:

These metrics help evaluate model performance for each career class.

Final Conclusion

Key findings from the project:

	1.	The dataset contains 1000 rows and 13 columns with no missing values.

	2.	There were 300 duplicate rows, indicating possible synthetic or templated data generation.

	3.	Most categorical variables are fairly balanced, reducing class imbalance problems.

	4.	Numeric columns such as Weight and Threshold have low variability, meaning they may have limited predictive power.

	5.	A Logistic Regression model was trained to predict careers based on skill-related features.
  
	6.	The model achieved moderate accuracy, demonstrating that career roles can be predicted from skill attributes.
"""

import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns

from sklearn.model_selection import train_test_split
from sklearn.preprocessing import LabelEncoder
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import accuracy_score, confusion_matrix, classification_report

"""Skill Scoring Model"""

def calculate_skill_score(user_proficiencies, career_name, dataframe):
  """
user_proficiencies: Dictionary { 'Skill Name': score_out_of_100 }
career_name: The target role (e.g., 'Data Engineer')
dataframe: Your cleaned 'df'
  """
  # Filter dataset for the specific career
  career_reqs = dataframe[dataframe['Career'] == career_name].copy()
  if career_reqs.empty:
    return "Career not found in dataset. "
  total_weighted_score = 0
  max_possible_weight = career_reqs['Weight'].sum()
  for _, row in career_reqs.iterrows():
    skill = row['Skill/Sub-skill']
    weight = row['Weight']
    threshold = row['Threshold (0-100)']
    # Get user score or assume 0 if not provided
    user_score = user_proficiencies.get(skill, 0)
    # Scoring logic: User score relative to the threshold, multiplied by weight
    # If user exceeds threshold, they get full weight for that skill
    actual_score = min(user_score / threshold, 1.0) * weight if threshold > 0 else 0
    total_weighted_score += actual_score
  # Normalize to a 0-100 scale
  final_readiness = (total_weighted_score / max_possible_weight) * 100
  return round(final_readiness, 2)
# Example Usage:
# my_skills = {'REST Concepts': 80, 'Unit
# Testing': 50}
# score = calculate_skill_score(my_skills,'Data Engineer', df )
# print(f"Readiness Score: {score}%")

"""Implement Skill Gap Analysis"""

def implement_skill_gap_analysis(user_proficiencies, target_career, dataframe):
    career_requirements = dataframe[dataframe['Career'] == target_career]
    pass

my_user_skills = {
    'REST Concepts': 75,
    'Debugging Basics': 80,
    'Unit Testing': 60,
    'API Consumption': 70,
    'Data Modeling': 65
}

target_career = 'Data Engineer'

# Ensure the dataframe 'df' is available and cleaned
# If you've been using df_model, remember it's encoded. For skill names, use the original df.

# Call the function
# Assuming implement_skill_gap_analysis is defined in a previous cell
# For now, let's just demonstrate how to call it, even if it's not fully implemented yet
# The function currently only defines 'career_requirements' and passes.

implement_skill_gap_analysis(my_user_skills, target_career, df)

print(f"Function 'implement_skill_gap_analysis' called for {target_career} with provided skills.")

def skill_score(rows):
      return (rows['Skill/Sub-skill'] + rows['Threshold (0-100)'] / 2)

def calculate_skill_score(user_proficiencies, career_name, dataframe):
    pass