Target lora modules identity for Llama 2 7B on Amazon SageMaker Studio C

Thank you Giovanni Lauria for your answer.

These are some useful suggestions, but I have tried most of them already and couldn't find the information.

Also, this answer has been clearly copy/pasted directly from an LLM (GPT or maybe the one used internally within Amazon). I did check some suggestions from GPT before posting here, which were the same as the ones suggested here but couldn't get anywhere.

Is there no way to find out the actual Amazon documentation that specifies this information? Or some kind of metadata that I could check?

Thank you.

Try the following. to investigate the behavior of LoRA in Llama 2 7B models during October/November 2023, even in the absence of explicit documentation or access to historical configurations.

1. Literature Review: Begin by conducting a thorough literature review on LoRA algorithm implementations, particularly focusing on papers, articles, or official documentation released around the timeframe of October/November 2023. Look for any discussions, publications, or updates related to the integration of LoRA with Llama 2 7B models or similar architectures.

2. Documentation Analysis: Scrutinize any available documentation from AWS, SageMaker, or related repositories. Pay close attention to release notes, version updates, or changelogs that might provide insights into the introduction or modification of hyperparameters, including the default settings for LoRA-related parameters.

3. Experiment Replication: Attempt to replicate the training environment and experimental conditions prevalent during October/November 2023. If feasible, access historical training data, compute resources, and training scripts used during that time. Reproduce the training process for Llama 2 7B models and meticulously document any differences or nuances observed compared to current configurations.

4. Parameter Sensitivity Analysis: Conduct a parameter sensitivity analysis to investigate the impact of varying hyperparameters, including those related to LoRA, on model performance. Systematically modify hyperparameters within a reasonable range and monitor changes in model behavior, such as convergence speed, accuracy, or loss curves. This analysis could help infer the default settings for LoRA in absence of explicit documentation.

5. Cross-Validation Studies: Employ cross-validation techniques to assess the robustness and generalization performance of Llama 2 7B models trained with different hyperparameter configurations, including variations in LoRA-related settings. Compare model performance metrics across different settings to discern patterns or dependencies that could hint at the default behavior of LoRA.

6. Statistical Inference: Utilize statistical inference methods to analyze experimental results and draw meaningful conclusions about the effects of hyperparameters on model behavior. Employ hypothesis testing, analysis of variance (ANOVA), or regression analysis to quantify the significance of variations in LoRA-related parameters and their impact on model outcomes.

7. Peer Review and Collaboration: Engage with peers, researchers, or practitioners in the field of machine learning and natural language processing. Share your findings, methodologies, and insights related to the investigation of LoRA in Llama 2 7B models. Collaborate with others to validate results, exchange perspectives, and collectively advance understanding in this domain.