To customize your babel configuration, simply create a .babelrc with the desired configuration. Do not forget to add next/babel preset into your configuration.
{"presets": ["next/babel" ]}
Please refer to this documentation for customizing babel configuration.