Upload README.md

README.md

@@ -1,3 +1,48 @@
----
-license: mit
----
+# GH29BERT
+This repository contains the code and testing sequence data for reproduce the prediction results for GH29BERT, a protein functional cluster prediction model devised for GH29 family sequences. It is trained based on a semi-supervised deep learning method with:
+	- a. 34,258 unlabeled and non-redundant GH29 sequences (i.e., unlabelled data) extracted from CAZy and Interpro databases and
+	- b. 2,796 labelled sequences with 45 cluster classes based on a thorough SSN analysis.
+Specifically, the reproducible testing materials (code and data) on following two types of GH29 sequences used in submitted manuscript are provided, including:
+	- 559 labelled GH29 testing sequences (2,796 labelled data with a random 80%-20% split for training and testing), see file `data/test.fasta`
+	- 15 held-out characterized sequences that was excluded from both pre-training and task-training, see file `data/15_seq_for-test.fasta`
+## Interactive deployment of GH29BERT for prediction testing
+	GH29BERT model is also accessible through a friendly user-interface on HuggingFace: https://huggingface.co/spaces/Oiliver/GH29BERT. It is easier to test the above provided GH29 sequences or your custom GH29 sequence using this web tool.
+## Prerequisites
+	 ### Repository download
+		To get started, clone this repository, e.g., execute the following in the terminal: `git clone https://github.com/ke-xing/GH29BERT.git`
+	 ### Environment preparation
+		Please check all the useful packages in the file **environment.yml**.   
+		  With the help of [Conda](https://docs.conda.io/projects/conda/en/stable/user-guide/getting-started.html), run `conda env create --file environment.yml` to create an independent environment for implementing the testing  
+	 ### Model parameter download
+		Due to the limit of single file size of GitHub repository, we upload the model parameter files at [Zenodo open repository](https://zenodo.org/records/10614689)
+			- GH29BERT
+				  ```python
+				  # Load GH29BERT pre-trained model
+				  GH29BERT=torch.load('transformer1500_95p_500.pt')
+				  GH29BERT=GH29BERT.module
+				  GH29BERT=GH29BERT.to('cuda:0')
+				  # Load GH29BERT task model
+				  downstream_GH29BERT=torch.load('down_model_500_kfold1.pt').to('cuda:0')
+				  ```
+			- ProtT5-XL
+				- Reproducing prediction testing based on pre-trained ProtT5-XL requires installing extra dependency libraries:
+					  ```
+					  pip install torch
+					  pip install transformers
+					  pip install sentencepiece
+					  ```
+					- For more details, please follow the instructions of [ProtTrans](https://ieeexplore.ieee.org/document/9477085) repository from [github](https://github.com/agemagician/ProtTrans/?tab=readme-ov-file).
+				  ```python
+				  from transformers import T5Tokenizer, T5EncoderModel
+				  
+				  # Load ProtT5_XL pre-trained model
+				  ProtT5_XL=T5EncoderModel.from_pretrained("Rostlab/prot_t5_xl_half_uniref50-enc",cache_dir='./').to('cuda:0')
+				  # Load ProtT5_XL task model
+				  downstream_ProtT5_XL=torch.load('down_model_500_kfold1.pt').to('cuda:0')
+				  ```
+- ## Cluster prediction
+	 Run `python python test.py` for predicting the fasta data. Model and data loading directory should be adjusted if need.
+- ## Representation visualization
+	The visualization of GH29 representations with GH29BERT or other pre-training models can be implemented through `python visualization by UMAP.py` for obtaining the dimension-reduced intermediate representations and run `python figure1.py figure2.py` to get the visualization map.
+- ## Code for model training
+	We also provide the model training code for pre-training  and downstream task-training. Run `python Pretrain/transformer/transformer_train.py` for GH29BERT model pre-training. Run`python classification/downstream_embedding.py` for loading the pre-trained model parameters and the embedding data(.npz) preparing for the task-training, and then run `python classification/downstream_train.py` for cluster prediction for task-training.