Deep learning that scales: leveraging compute and data

Deep learning has revolutionized the field of artificial intelligence in the past decade. Although the development of these techniques spans over several years, the recent advent of deep learning is explained by an increased availability of data and compute that have unlocked the potential of deep n...

Descripción completa

Detalles Bibliográficos
Autor: Campos Camúñez, Víctor
Tipo de recurso: tesis doctoral
Estado:Versión publicada
Fecha de publicación:2020
País:España
Institución:CBUC, CESCA
Repositorio:TDR. Tesis Doctorales en Red
OAI Identifier:oai:www.tdx.cat:10803/670372
Acceso en línea:http://hdl.handle.net/10803/670372
https://dx.doi.org/10.5821/dissertation-2117-335426
Access Level:acceso abierto
Palabra clave:Àrees temàtiques de la UPC::Informàtica
004
id ES_577dd3780669cd1facd63a33d22dedcc
oai_identifier_str oai:www.tdx.cat:10803/670372
network_acronym_str ES
network_name_str España
repository_id_str
spelling Deep learning that scales: leveraging compute and dataCampos Camúñez, VíctorÀrees temàtiques de la UPC::Informàtica004Deep learning has revolutionized the field of artificial intelligence in the past decade. Although the development of these techniques spans over several years, the recent advent of deep learning is explained by an increased availability of data and compute that have unlocked the potential of deep neural networks. They have become ubiquitous in domains such as natural language processing, computer vision, speech processing, and control, where enough training data is available. Recent years have seen continuous progress driven by ever-growing neural networks that benefited from large amounts of data and computing power. This thesis is motivated by the observation that scale is one of the key factors driving progress in deep learning research, and aims at devising deep learning methods that scale gracefully with the available data and compute. We narrow down this scope into two main research directions. The first of them is concerned with designing hardware-aware methods which can make the most of the computing resources in current high performance computing facilities. We then study bottlenecks preventing existing methods from scaling up as more data becomes available, providing solutions that contribute towards enabling training of more complex models. This dissertation studies the aforementioned research questions for two different learning paradigms, each with its own algorithmic and computational characteristics. The first part of this thesis studies the paradigm where the model needs to learn from a collection of examples, extracting as much information as possible from the given data. The second part is concerned with training agents that learn by interacting with a simulated environment, which introduces unique challenges such as efficient exploration and simulation.DOCTORAT EN ARQUITECTURA DE COMPUTADORS (Pla 2012)Universitat Politècnica de CatalunyaTorres, Jordi (Torres Viñals)Giró i Nieto, XavierUniversitat Politècnica de Catalunya. Departament d'Arquitectura de Computadors202120212020info:eu-repo/semantics/doctoralThesisinfo:eu-repo/semantics/publishedVersion156 p.application/pdfapplication/pdfhttp://hdl.handle.net/10803/670372https://dx.doi.org/10.5821/dissertation-2117-335426TDX (Tesis Doctorals en Xarxa)reponame:TDR. Tesis Doctorales en Redinstname:CBUC, CESCAInglésL'accés als continguts d'aquesta tesi queda condicionat a l'acceptació de les condicions d'ús establertes per la següent llicència Creative Commons: http://creativecommons.org/licenses/by-nc/4.0/http://creativecommons.org/licenses/by-nc/4.0/info:eu-repo/semantics/openAccessoai:www.tdx.cat:10803/6703722026-06-14T12:46:07Z
dc.title.none.fl_str_mv Deep learning that scales: leveraging compute and data
title Deep learning that scales: leveraging compute and data
spellingShingle Deep learning that scales: leveraging compute and data
Campos Camúñez, Víctor
Àrees temàtiques de la UPC::Informàtica
004
title_short Deep learning that scales: leveraging compute and data
title_full Deep learning that scales: leveraging compute and data
title_fullStr Deep learning that scales: leveraging compute and data
title_full_unstemmed Deep learning that scales: leveraging compute and data
title_sort Deep learning that scales: leveraging compute and data
dc.creator.none.fl_str_mv Campos Camúñez, Víctor
author Campos Camúñez, Víctor
author_facet Campos Camúñez, Víctor
author_role author
dc.contributor.none.fl_str_mv Torres, Jordi (Torres Viñals)
Giró i Nieto, Xavier
Universitat Politècnica de Catalunya. Departament d'Arquitectura de Computadors
dc.subject.none.fl_str_mv Àrees temàtiques de la UPC::Informàtica
004
topic Àrees temàtiques de la UPC::Informàtica
004
description Deep learning has revolutionized the field of artificial intelligence in the past decade. Although the development of these techniques spans over several years, the recent advent of deep learning is explained by an increased availability of data and compute that have unlocked the potential of deep neural networks. They have become ubiquitous in domains such as natural language processing, computer vision, speech processing, and control, where enough training data is available. Recent years have seen continuous progress driven by ever-growing neural networks that benefited from large amounts of data and computing power. This thesis is motivated by the observation that scale is one of the key factors driving progress in deep learning research, and aims at devising deep learning methods that scale gracefully with the available data and compute. We narrow down this scope into two main research directions. The first of them is concerned with designing hardware-aware methods which can make the most of the computing resources in current high performance computing facilities. We then study bottlenecks preventing existing methods from scaling up as more data becomes available, providing solutions that contribute towards enabling training of more complex models. This dissertation studies the aforementioned research questions for two different learning paradigms, each with its own algorithmic and computational characteristics. The first part of this thesis studies the paradigm where the model needs to learn from a collection of examples, extracting as much information as possible from the given data. The second part is concerned with training agents that learn by interacting with a simulated environment, which introduces unique challenges such as efficient exploration and simulation.
publishDate 2020
dc.date.none.fl_str_mv 2020
2021
2021
dc.type.none.fl_str_mv info:eu-repo/semantics/doctoralThesis
info:eu-repo/semantics/publishedVersion
format doctoralThesis
status_str publishedVersion
dc.identifier.none.fl_str_mv http://hdl.handle.net/10803/670372
https://dx.doi.org/10.5821/dissertation-2117-335426
url http://hdl.handle.net/10803/670372
https://dx.doi.org/10.5821/dissertation-2117-335426
dc.language.none.fl_str_mv Inglés
language_invalid_str_mv Inglés
dc.rights.none.fl_str_mv http://creativecommons.org/licenses/by-nc/4.0/
info:eu-repo/semantics/openAccess
rights_invalid_str_mv http://creativecommons.org/licenses/by-nc/4.0/
eu_rights_str_mv openAccess
dc.format.none.fl_str_mv 156 p.
application/pdf
application/pdf
dc.publisher.none.fl_str_mv Universitat Politècnica de Catalunya
publisher.none.fl_str_mv Universitat Politècnica de Catalunya
dc.source.none.fl_str_mv TDX (Tesis Doctorals en Xarxa)
reponame:TDR. Tesis Doctorales en Red
instname:CBUC, CESCA
instname_str CBUC, CESCA
reponame_str TDR. Tesis Doctorales en Red
collection TDR. Tesis Doctorales en Red
repository.name.fl_str_mv
repository.mail.fl_str_mv
_version_ 1869408455906820096
score 15,300719

Similares en LA Referencia