Whisk the sugar, cornstarch, and salt in a medium saucepan. In a separate bowl, whisk the egg yolks with about 1/2 cup of the milk until smooth.
Whisk the remaining milk into the saucepan. Cook over medium heat, whisking constantly, until steaming. Slowly whisk about 1 cup of the hot milk mixture into the yolks to temper them.
Pour the yolk mixture back into the saucepan. Cook, whisking constantly, until the custard thickens and reaches 160°F on an instant-read thermometer, about 3 to 5 minutes. Remove from heat and whisk in the butter and vanilla.
Transfer the custard to a shallow bowl. Press plastic wrap directly onto the surface and chill until cold, at least 2 hours.
Beat the cold heavy cream and powdered sugar to soft peaks. Fold about one-third of the whipped cream into the chilled custard to lighten it.
In a 3-quart dish, arrange a layer of vanilla wafers, followed by banana slices and custard. Repeat the layers, finishing with custard. Cover and refrigerate for at least 4 hours so the wafers soften.
Spread the remaining whipped cream over the pudding before serving. Garnish with extra wafer crumbs if desired; keep refrigerated.