Overview

ericpolewski/ASTS-PFAF is an experimental fine-tuned language model developed by ericpolewski. It was created as part of a challenge to evaluate a specific dataset's impact on model performance, particularly in the context of stock market prediction for AST Spacemobile. The model integrates a public dataset with the creator's own collected news and financial data related to AST Spacemobile.

Key Characteristics

• Experimental Fine-tuning: The model was fine-tuned by combining a public dataset (from a challenge) with proprietary data on AST Spacemobile.
• Stock Market Prediction Focus: The primary, albeit skeptical, goal was to create a bot capable of predicting stock market movements for a specific pre-revenue company.
• Dataset Integration: The creator manually combined and formatted data into the Alpaca instruct/response format, noting the importance of randomization to prevent loss explosion during training.
• Training Environment: Fine-tuning was performed on a GPTQ model, chosen for faster loading times, rather than an FP16 model.
• Benchmark Approach: Performance evaluation relies on external leaderboards (e.g., for GSM8K scores) rather than local benchmarks.

Intended Use Cases

• Dataset Evaluation: Primarily serves as a testbed for evaluating the effectiveness of a specific dataset in improving language model performance.
• Speculative Financial Analysis: An attempt to create a bot that can provide insights or predictions regarding the stock market performance of AST Spacemobile, though the creator remains highly skeptical of its actual predictive power.
• Research into Fine-tuning: Demonstrates a practical, albeit unconventional, approach to fine-tuning with mixed datasets and the challenges involved.