Goliaths are slow enough already. Unwieldy (the heavy bolter) hits them hard. Get the tyrant a much cheaper and mobile bolter instead. You need more free credits for bodies. With 3 S4 punches Forge tyrants have more than enough melee power to afford not to have a melee weapon to start with so a Bolter Tyrant is rounded enough to start with while still being flexible.
Bullies are just as effective in melee as Bruisers with the same WS, S and A so if that's their role I would just use them instead. The Fast learner rule quickly and directly gives advances to boost the BS for secondary shooting and your Cool stat if that is an issue but your Bully doesn't have a bad value there. You can save credits there. Like the tyrant they are still S4, and above average S wounding most (T3) on 3+. To hit initial gangers and juves, which is really a melee ganger or juves primary target, that punch is entirely enough to start with.
I would avoid the fighting knife for Goliaths, even at the start, and when starting out just punching can suffice. You really just lack the speed and smaller size to make use of Backstab. Your opponents unless they are similar slow can almost guarantee to be facing you so deny you the S bonus. Yes they have AP-1. So does the Brute cleaver you have access to. And it gives +1 to hit in melee. And has a chance of denying weapon use in opponents Reaction attacks.
With your bullies punch your way through a few juves in the first game, or two or three, and get them Brute Cleavers instead.
The single ranged option of the Stub gun is all you bullies have. At least it's cheap, decent pistol short range and a good short range accuracy bonus to mask your poor BS. And unlike others basic sidearm the auto pistol can take the Infra Sight (you can Trade Post for it as it is Wargear and you are not limited there). Don't use them in melee as a Goliath unless you have not got two close combat weapons to gain duel wielding. Your basic punch is better. A couple of Brute Cleavers only costs 40 credits and is much better for you. Don't forget to upgrade it with ammo types especially if you shell out on a Bullet Merchant hanger on. (Special Ammo are NOT weapons. They are additions to the weapon, you don't throw the shells, so you can Trade Post for the ones outside the fighter Entry). Static rounds are pretty great for stub guns.
As Bullies can do everything a Bruiser can in melee I would only use them as a ranged firing line. With the pricy long arms,

. The Stub cannon your "cheap" option costs 5 to 15 credits more than others basic Lasgun or autogun. For less range. At least it's S5 with a good ammo roll and potentially if it Knocks back D2. They give an initial kick but fall behind as they lack range and accuracy which keep the long arm useful at later games and have no AP or Ammo to keep up the punch against later foes.
Bruisers with Bolters however have weapons that have good Range Damage Ap and Rapid fire. Alone they can fill in for a special weapon damage potential in a pinch. Two of them can eclipse a single special weapon for less cost. With the addition of TP special ammos you can give them a improved Frag grenade like Blast option or the awesome Gunk bolts to improve the utility.
One bolter brusier is well worth the investment early on. And a good option to copy later.
Shotguns of any type are really the darling ganger weapon of necromunda. And Goliaths get the Combat version at a discount. Alone I prefer the Combat shotgun shredder ammo over the Flamer with the mass potential of scatter shot. It's salvo ammo is excellent at close order firefights. Then you can add Firestorm ammo which, Ap aside, is better than the heavy Flamer as it has no Unwieldy. Give it Shatter shells and it gains a extended range option Blast attack. The 30 credit increase in price over the standard shotgun is worth it long term but you could start with a standard one and add the combat shotgun later for the template options (with or without modeling a Super Shotgun model).
Either a Bolter or Shotgun bruiser can easily take the place of a true special weapon specialist brusier. They only miss out on choosing their advances.
Your bread and butter ranged weapons, the stubgun, Bolter and shotguns are decent by themselves but are rapidly upgradable with Special ammo.
Get a Bullet Merchant, maybe a second hanger on after the Rouge doc.
His High Calibre Hookup provides Common access to any special ammo and changes Limited special ammo to Scarce. Across the whole gang. No rebuying Dudums, Firestorms, Shatter shells, Gunk shells or Static rounds.
If you have a gang with weapons that use alternative ammo the Bullet Merchant is the best Hanger on in the game.
Goliaths are one of those gangs.