Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add Benchmarking a foundational cell model for post-perturbation RNAseq prediction #25

Closed
szalata opened this issue Oct 16, 2024 · 4 comments

Comments

@szalata
Copy link
Collaborator

szalata commented Oct 16, 2024

https://www.biorxiv.org/content/biorxiv/early/2024/10/01/2024.09.30.615843.full.pdf

@geroldcsendes
Copy link
Contributor

Hi, thanks for your interest in our research!

In summary, we found:

  • Simple baseline models can outperform scGPT on perturbational downstream tasks.
  • Incorporating prior knowledge features, such as GO terms of perturbed proteins, can improve the performance of baseline models.
  • The most widely used benchmarking datasets contain significant biases, making them suboptimal for evaluation.
  • Models do not appear to benefit significantly from single-cell data, as those trained on bulk data can match or even outperform models trained on single-cell data.

Let me know if you need further information from us.

@Paulos2411
Copy link
Collaborator

Paulos2411 commented Jan 14, 2025

Hi Gerold,

thanks for your summary. Could you perhaps summarize the information in a way as it has been done on the website for your specific case.

https://theislab.github.io/single-cell-transformer-papers/

I will then add them or feel free to create a PR.

Thanks :)

@geroldcsendes
Copy link
Contributor

Hi, I made the PR here: #74

@Paulos2411
Copy link
Collaborator

Great, thank you.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

3 participants