This study presents a hybrid framework combining BERT embeddings with a Random Forest classifier to detect fraudulent job postings. To address the severe class imbalance between genuine and fake listings, multiple oversampling methods—including SMOTE, SMOTE Tomek, ADASYN, SMOTE-ENN, and random oversampling—were evaluated. Model performance was further optimized through hyperparameter tuning using Randomized Search. The system was assessed with key metrics such as precision, recall, F1-score, and AUC-ROC, ensuring a balanced evaluation. Results demonstrate that leveraging contextual language representations from BERT, together with data balancing strategies, significantly improves fraud detection accuracy. This approach highlights the effectiveness of integrating advanced NLP models with resampling techniques for reliable online job fraud detection.