Tom Joy

Tom Joy

Entrepreneur (AI), Co-founder of GirlsWhoML, Prev TVG (PhD), Meta (intern), Five AI (Researcher)

Sitemap

A list of all the posts and pages found on the site. For you robots out there is an XML version available for digesting as well.

Pages

Posts

Generative Models in AI, GANs, VAEs, INNs

13 minute read

Published:

Generative Models in AI, GANs, VAEs, INNs

The main goal of generative model is the ‘generate’ some new data, unlike in classification where we have a fixed set of classes we want to predict. It’s basically producing rather than predicting. There are four main types of generative models: VAEs, GANs, Normalizing Flows, Invertible Neural Networks and Diffusion Models (for next time), and they all try and learn a model which captures the underlying structure of the data.

Intro to Deep Learning 2 – Loss Landscapes, Trainig Dynamics, and Optimizers

9 minute read

Published:

Loss Landscapes

In the last post we spoke about walking down the hill to find the location of the lowest point. This is a great analogy for training a neural network, we want to find the parameters that minimize the loss function. You can think of the loss function as the height of the hill, and the parameters as the coordinates of the hill. This landscape is called the loss landscape, and we talk alot about it in the deep learning community. Specifically in relation to valleys, local minima, saddle points, and flat regions. Whenever you hear these terms, just think about hills and valleys.

Intro to Deep Learning – Training, Neural Networks, and Feature Representations

9 minute read

Published:

Intro

Basically every machine learning AI problem is now solved using deep learning. Deep learning is the approach of using neural networks to learn from data, in this approach we don’t need to decide on what the model (neural network) should do, we just need to give it the data and the model will learn to do the task.

portfolio

publications

Real-Time Highly Accurate Dense Depth on a Power Budget using an FPGA-CPU Hybrid SoC

Published in IEEE Transactions on Circuits and Systems II: Express Briefs, 2019

@article{rahnama2019real,
  title={Real-Time Highly Accurate Dense Depth on a Power Budget using an FPGA-CPU Hybrid SoC},
  author={Rahnama, Oscar and Cavallari, Tommaso and Golodetz, Stuart and Tonioni, Alessio and Joy, Thomas and Di Stefano, Luigi and Walker, Simon and Torr, Philip HS},
  journal={IEEE Transactions on Circuits and Systems II: Express Briefs},
  year={2019},
  publisher={IEEE}
}

Download here

Learning to Adapt for Stereo

Published in CVPR, 2019

@article{tonioni2019learning,
  title={Learning to Adapt for Stereo},
  author={Tonioni, A and Rahnama, O and Joy, T and
         Di Stefano, L and Ajanthan, T and
         Torr, P.H.S},
  year={2019},
  journal={CVPR},
}

Download here

Efficient Relaxations for Dense CRFs with Sparse Higher-Order Potentials

Published in SIAM J. Imaging Sci. 287--318 Vol 12, 2019

@article{joy2019densecrf,
	author = {Joy, T and Desmaison, A and Ajanthan, T and Bunel, R and Salzmann, M and Kohli, P and Torr, P.H.S and Kumar, M},
	journal = {SIAM J. Imaging Sci.},
	number = {1},
	pages = {287--318},
	title = {Efficient Relaxations for Dense CRFs with Sparse Higher-Order Potentials},
	volume = {12},
	year = {2019}
}

Download here

Capturing Label Characteristics in VAEs

Published in ICLR 2021, 2021

@article{joy2020capturing,
  title={Capturing Label Characteristics in VAEs},
  author={Joy, Tom and Schmon, Sebastian and Torr, Philip H.S. and Siddharth, N and Rainforth, Tom},
  journal={International Conference on Learning Representations},
  year={2020}
}

Download here

Learning Multimodal VAEs through Mutual Supervision

Published in ICLR, 2022

@article{joy2021learning,
  title={Learning Multimodal VAEs through Mutual Supervision},
  author={Joy, Tom and Shi, Yuge and Torr, Philip H.S. and Rainforth, Tom and Schmon, Sebastian M and Siddharth, N},
  journal={ICLR},
  year={2022}
}

Download here

What makes and breaks safety fine-tuning? a mechanistic study

Published in NeurIPS, 2024

@article{jain2024makes,
  title={What makes and breaks safety fine-tuning? a mechanistic study},
  author={Jain, Samyak and Lubana, Ekdeep Singh and Oksuz, Kemal and Joy, Tom and Torr, Philip HS and Sanyal, Amartya and Dokania, Puneet K},
  booktitle={NeurIPS},
  year={2024}
}

Download here

talks

teaching

Teaching experience 1

Undergraduate course, University 1, Department, 2014

This is a description of a teaching experience. You can use markdown like any other post.

Teaching experience 2

Workshop, University 1, Department, 2015

This is a description of a teaching experience. You can use markdown like any other post.