Neural networks with low-resolution parameters
Carregando...
Data
Data de publicação
Autores IPEN
Orientador
Título da Revista
ISSN da Revista
Título do Volume
É parte de
É parte de
É parte de
Neural Networks
Exportar
Resumo
The expanding scale of large neural network models introduces significant challenges, driving efforts to
reduce memory usage and enhance computational efficiency. Such measures are crucial to ensure the practical
implementation and effective application of these sophisticated models across a wide array of use cases. This
study examines the impact of parameter bit precision on model performance compared to standard 32-bit
models, with a focus on multiclass object classification in images. The models analyzed include those with
fully connected layers, convolutional layers, and transformer blocks, with model weight resolution ranging
from 1 bit to 4.08 bits. The findings indicate that models with lower parameter bit precision achieve results
comparable to 32-bit models, showing promise for use in memory-constrained devices. While low-resolution
models with a small number of parameters require more training epochs to achieve accuracy comparable
to 32-bit models, those with a large number of parameters achieve similar performance within the same
number of epochs. Additionally, data augmentation can destabilize training in low-resolution models, but
including zero as a potential value in the weight parameters helps maintain stability and prevents performance
degradation. Overall, 2.32-bit weights offer the optimal balance of memory reduction, performance, and
efficiency. However, further research should explore other dataset types and more complex and larger
models. These findings suggest a potential new era for optimized neural network models with reduced
memory requirements and improved computational efficiency, though advancements in dedicated hardware
are necessary to fully realize this potential.
Como referenciar
CABRAL, EDUARDO L.L.; DRIEMEIER, LARISSA. Neural networks with low-resolution parameters. Neural Networks, v. 191, p. 1-19, 2025. DOI: 10.1016/j.neunet.2025.107763. Disponível em: https://repositorio.ipen.br/handle/123456789/49428. Acesso em: 20 Mar 2026.
Esta referência é gerada automaticamente de acordo com as normas do estilo IPEN/SP (ABNT NBR 6023) e recomenda-se uma verificação final e ajustes caso necessário.