XEUIPR commited on
Commit
f212e20
·
verified ·
1 Parent(s): a73a7f1

Update README.md

Browse files
Files changed (1) hide show
  1. README.md +3 -95
README.md CHANGED
@@ -18,101 +18,9 @@ Java-Code-Large is a large-scale corpus of publicly available Java source code c
18
  By providing a high-volume, language-specific corpus, Java-Code-Large enables systematic experimentation in Java-focused model training, domain adaptation, and downstream code understanding tasks.
19
 
20
 
21
- **1. Introduction**
22
 
23
- Large-scale code corpora have become fundamental resources for training and evaluating machine learning models for code-related tasks. While multilingual code datasets exist, there is increasing interest in language-specialized corpora to:
24
 
25
- - Improve domain-specific performance
26
-
27
- - Reduce cross-language noise
28
-
29
- - Enable controlled experimental settings
30
-
31
- - Support Java-specific tooling and research
32
-
33
- Java-Code-Large addresses this need by providing a dedicated Java-only dataset at substantial scale.
34
-
35
-
36
- **2. Dataset Composition**
37
-
38
- Programming Language: Java
39
-
40
- File Count: 15M+ Java files
41
-
42
- File Format: .jsonl
43
-
44
- Content Types:
45
-
46
- - Classes
47
-
48
- - Interfaces
49
-
50
- - Enums
51
-
52
- - Methods
53
-
54
- - Annotations
55
-
56
- - JavaDoc comments
57
-
58
- - Exception handling structures
59
-
60
- - Generics and concurrency constructs
61
-
62
- The dataset consists of source code extracted from publicly accessible open-source repositories.
63
-
64
-
65
- **3. Intended Research Applications**
66
-
67
-
68
- 3.1 Pretraining
69
-
70
- - Training code foundation models from scratch
71
-
72
- - Continued pretraining of existing LLMs
73
-
74
- - Java-specialized language modeling
75
-
76
-
77
-
78
- 3.2 Fine-Tuning and Adaptation
79
-
80
- - Code completion systems
81
-
82
- - Automated refactoring tools
83
-
84
- - IDE copilots
85
-
86
- - Java-specific conversational assistants
87
-
88
-
89
-
90
- 3.3 Code Intelligence Tasks
91
-
92
- - Code summarization
93
-
94
- - Code-to-text generation
95
-
96
- - Bug detection
97
-
98
- - Vulnerability detection
99
-
100
- - Clone detection
101
-
102
- - Code similarity modeling
103
-
104
- - Static and structural analysis
105
-
106
-
107
-
108
- 3.4 Software Engineering Research
109
-
110
- - Empirical studies of Java programming patterns
111
-
112
- - Tokenization and AST modeling experiments
113
-
114
-
115
-
116
-
117
- Thanks to open source community for all the guidance & support!!
118
 
 
 
18
  By providing a high-volume, language-specific corpus, Java-Code-Large enables systematic experimentation in Java-focused model training, domain adaptation, and downstream code understanding tasks.
19
 
20
 
21
+ -----
22
 
 
23
 
24
+ Duplicated from [ajibawa-2023/Java-Code-Large](https://huggingface.co/datasets/ajibawa-2023/Java-Code-Large), where the only difference is that the "language" column containing only "java" is removed.
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
25
 
26
+ Dataset updated using huggingface_hub datasets on colab.