Missing keys
Loading ZImage model
Loading transformer
Loading checkpoint shards: 100%|##########| 3/3 [00:12<00:00, 4.06s/it]
Loading assistant LoRA
create LoRA network. base dim (rank): 96, alpha: 96
neuron dropout: p=None, rank dropout: p=None, module dropout: p=None
create LoRA for Text Encoder: 0 modules.
create LoRA for U-Net: 276 modules.
enable LoRA for U-Net
Merging in assistant LoRA
Missing keys: ['transformer$$cap_pad_token.lora_down.weight', 'transformer$$cap_pad_token.lora_up.weight', 'transformer$$context_refiner$$0$$attention$$out.lora_down.weight', 'transformer$$context_refiner$$0$$attention$$out.lora_up.weight', 'transformer$$context_refiner$$0$$attention$$qkv.lora_down.weight', 'transformer$$context_refiner$$0$$attention$$qkv.lora_up.weight', 'transformer$$context_refiner$$1$$attention$$out.lora_down.weight', 'transformer$$context_refiner$$1$$attention$$out.lora_up.weight', 'transformer$$context_refiner$$1$$attention$$qkv.lora_down.weight', 'transformer$$context_refiner$$1$$attention$$qkv.lora_up.weight', 'transformer$$final_layer$$adaLN_modulation$$1.lora_down.weight', 'transformer$$final_layer$$adaLN_modulation$$1.lora_up.weight', 'transformer$$final_layer$$linear.lora_down.weight', 'transformer$$final_layer$$linear.lora_up.weight', 'transformer$$layers$$0$$attention$$out.lora_down.weight', 'transformer$$layers$$0$$attention$$out.lora_up.weight', 'transformer$$layers$$0$$attention$$qkv.lora_down.weight', 'transformer$$layers$$0$$attention$$qkv.lora_up.weight', 'transformer$$layers$$1$$attention$$out.lora_down.weight', 'transformer$$layers$$1$$attention$$out.lora_up.weight', 'transformer$$layers$$1$$attention$$qkv.lora_down.weight', 'transformer$$layers$$1$$attention$$qkv.lora_up.weight', 'transformer$$layers$$10$$attention$$out.lora_down.weight', 'transformer$$layers$$10$$attention$$out.lora_up.weight', 'transformer$$layers$$10$$attention$$qkv.lora_down.weight', 'transformer$$layers$$10$$attention$$qkv.lora_up.weight', 'transformer$$layers$$11$$attention$$out.lora_down.weight', 'transformer$$layers$$11$$attention$$out.lora_up.weight', 'transformer$$layers$$11$$attention$$qkv.lora_down.weight', 'transformer$$layers$$11$$attention$$qkv.lora_up.weight', 'transformer$$layers$$12$$attention$$out.lora_down.weight', 'transformer$$layers$$12$$attention$$out.lora_up.weight', 'transformer$$layers$$12$$attention$$qkv.lora_down.weight', 'transformer$$layers$$12$$attention$$qkv.lora_up.weight', 'transformer$$layers$$13$$attention$$out.lora_down.weight', 'transformer$$layers$$13$$attention$$out.lora_up.weight', 'transformer$$layers$$13$$attention$$qkv.lora_down.weight', 'transformer$$layers$$13$$attention$$qkv.lora_up.weight', 'transformer$$layers$$14$$attention$$out.lora_down.weight', 'transformer$$layers$$14$$attention$$out.lora_up.weight', 'transformer$$layers$$14$$attention$$qkv.lora_down.weight', 'transformer$$layers$$14$$attention$$qkv.lora_up.weight', 'transformer$$layers$$15$$attention$$out.lora_down.weight', 'transformer$$layers$$15$$attention$$out.lora_up.weight', 'transformer$$layers$$15$$attention$$qkv.lora_down.weight', 'transformer$$layers$$15$$attention$$qkv.lora_up.weight', 'transformer$$layers$$16$$attention$$out.lora_down.weight', 'transformer$$layers$$16$$attention$$out.lora_up.weight', 'transformer$$layers$$16$$attention$$qkv.lora_down.weight', 'transformer$$layers$$16$$attention$$qkv.lora_up.weight', 'transformer$$layers$$17$$attention$$out.lora_down.weight', 'transformer$$layers$$17$$attention$$out.lora_up.weight', 'transformer$$layers$$17$$attention$$qkv.lora_down.weight', 'transformer$$layers$$17$$attention$$qkv.lora_up.weight', 'transformer$$layers$$18$$attention$$out.lora_down.weight', 'transformer$$layers$$18$$attention$$out.lora_up.weight', 'transformer$$layers$$18$$attention$$qkv.lora_down.weight', 'transformer$$layers$$18$$attention$$qkv.lora_up.weight', 'transformer$$layers$$19$$attention$$out.lora_down.weight', 'transformer$$layers$$19$$attention$$out.lora_up.weight', 'transformer$$layers$$19$$attention$$qkv.lora_down.weight', 'transformer$$layers$$19$$attention$$qkv.lora_up.weight', 'transformer$$layers$$2$$attention$$out.lora_down.weight', 'transformer$$layers$$2$$attention$$out.lora_up.weight', 'transformer$$layers$$2$$attention$$qkv.lora_down.weight', 'transformer$$layers$$2$$attention$$qkv.lora_up.weight', 'transformer$$layers$$20$$attention$$out.lora_down.weight', 'transformer$$layers$$20$$attention$$out.lora_up.weight', 'transformer$$layers$$20$$attention$$qkv.lora_down.weight', 'transformer$$layers$$20$$attention$$qkv.lora_up.weight', 'transformer$$layers$$21$$attention$$out.lora_down.weight', 'transformer$$layers$$21$$attention$$out.lora_up.weight', 'transformer$$layers$$21$$attention$$qkv.lora_down.weight', 'transformer$$layers$$21$$attention$$qkv.lora_up.weight', 'transformer$$layers$$22$$attention$$out.lora_down.weight', 'transformer$$layers$$22$$attention$$out.lora_up.weight', 'transformer$$layers$$22$$attention$$qkv.lora_down.weight', 'transformer$$layers$$22$$attention$$qkv.lora_up.weight', 'transformer$$layers$$23$$attention$$out.lora_down.weight', 'transformer$$layers$$23$$attention$$out.lora_up.weight', 'transformer$$layers$$23$$attention$$qkv.lora_down.weight', 'transformer$$layers$$23$$attention$$qkv.lora_up.weight', 'transformer$$layers$$24$$attention$$out.lora_down.weight', 'transformer$$layers$$24$$attention$$out.lora_up.weight', 'transformer$$layers$$24$$attention$$qkv.lora_down.weight', 'transformer$$layers$$24$$attention$$qkv.lora_up.weight', 'transformer$$layers$$25$$attention$$out.lora_down.weight', 'transformer$$layers$$25$$attention$$out.lora_up.weight', 'transformer$$layers$$25$$attention$$qkv.lora_down.weight', 'transformer$$layers$$25$$attention$$qkv.lora_up.weight', 'transformer$$layers$$26$$attention$$out.lora_down.weight', 'transformer$$layers$$26$$attention$$out.lora_up.weight', 'transformer$$layers$$26$$attention$$qkv.lora_down.weight', 'transformer$$layers$$26$$attention$$qkv.lora_up.weight', 'transformer$$layers$$27$$attention$$out.lora_down.weight', 'transformer$$layers$$27$$attention$$out.lora_up.weight', 'transformer$$layers$$27$$attention$$qkv.lora_down.weight', 'transformer$$layers$$27$$attention$$qkv.lora_up.weight', 'transformer$$layers$$28$$attention$$out.lora_down.weight', 'transformer$$layers$$28$$attention$$out.lora_up.weight', 'transformer$$layers$$28$$attention$$qkv.lora_down.weight', 'transformer$$layers$$28$$attention$$qkv.lora_up.weight', 'transformer$$layers$$29$$attention$$out.lora_down.weight', 'transformer$$layers$$29$$attention$$out.lora_up.weight', 'transformer$$layers$$29$$attention$$qkv.lora_down.weight', 'transformer$$layers$$29$$attention$$qkv.lora_up.weight', 'transformer$$layers$$3$$attention$$out.lora_down.weight', 'transformer$$layers$$3$$attention$$out.lora_up.weight', 'transformer$$layers$$3$$attention$$qkv.lora_down.weight', 'transformer$$layers$$3$$attention$$qkv.lora_up.weight', 'transformer$$layers$$4$$attention$$out.lora_down.weight', 'transformer$$layers$$4$$attention$$out.lora_up.weight', 'transformer$$layers$$4$$attention$$qkv.lora_down.weight', 'transformer$$layers$$4$$attention$$qkv.lora_up.weight', 'transformer$$layers$$5$$attention$$out.lora_down.weight', 'transformer$$layers$$5$$attention$$out.lora_up.weight', 'transformer$$layers$$5$$attention$$qkv.lora_down.weight', 'transformer$$layers$$5$$attention$$qkv.lora_up.weight', 'transformer$$layers$$6$$attention$$out.lora_down.weight', 'transformer$$layers$$6$$attention$$out.lora_up.weight', 'transformer$$layers$$6$$attention$$qkv.lora_down.weight', 'transformer$$layers$$6$$attention$$qkv.lora_up.weight', 'transformer$$layers$$7$$attention$$out.lora_down.weight', 'transformer$$layers$$7$$attention$$out.lora_up.weight', 'transformer$$layers$$7$$attention$$qkv.lora_down.weight', 'transformer$$layers$$7$$attention$$qkv.lora_up.weight', 'transformer$$layers$$8$$attention$$out.lora_down.weight', 'transformer$$layers$$8$$attention$$out.lora_up.weight', 'transformer$$layers$$8$$attention$$qkv.lora_down.weight', 'transformer$$layers$$8$$attention$$qkv.lora_up.weight', 'transformer$$layers$$9$$attention$$out.lora_down.weight', 'transformer$$layers$$9$$attention$$out.lora_up.weight', 'transformer$$layers$$9$$attention$$qkv.lora_down.weight', 'transformer$$layers$$9$$attention$$qkv.lora_up.weight', 'transformer$$noise_refiner$$0$$attention$$out.lora_down.weight', 'transformer$$noise_refiner$$0$$attention$$out.lora_up.weight', 'transformer$$noise_refiner$$0$$attention$$qkv.lora_down.weight', 'transformer$$noise_refiner$$0$$attention$$qkv.lora_up.weight', 'transformer$$noise_refiner$$1$$attention$$out.lora_down.weight', 'transformer$$noise_refiner$$1$$attention$$out.lora_up.weight', 'transformer$$noise_refiner$$1$$attention$$qkv.lora_down.weight', 'transformer$$noise_refiner$$1$$attention$$qkv.lora_up.weight', 'transformer$$x_embedder.lora_down.weight', 'transformer$$x_embedder.lora_up.weight', 'transformer$$x_pad_token.lora_down.weight', 'transformer$$x_pad_token.lora_up.weight']
Attempting to load with forced keymap
Missing keys: ['transformer$$cap_pad_token.lora_down.weight', 'transformer$$cap_pad_token.lora_up.weight', 'transformer$$context_refiner$$0$$attention$$out.lora_down.weight', 'transformer$$context_refiner$$0$$attention$$out.lora_up.weight', 'transformer$$context_refiner$$0$$attention$$qkv.lora_down.weight', 'transformer$$context_refiner$$0$$attention$$qkv.lora_up.weight', 'transformer$$context_refiner$$1$$attention$$out.lora_down.weight', 'transformer$$context_refiner$$1$$attention$$out.lora_up.weight', 'transformer$$context_refiner$$1$$attention$$qkv.lora_down.weight', 'transformer$$context_refiner$$1$$attention$$qkv.lora_up.weight', 'transformer$$final_layer$$adaLN_modulation$$1.lora_down.weight', 'transformer$$final_layer$$adaLN_modulation$$1.lora_up.weight', 'transformer$$final_layer$$linear.lora_down.weight', 'transformer$$final_layer$$linear.lora_up.weight', 'transformer$$layers$$0$$attention$$out.lora_down.weight', 'transformer$$layers$$0$$attention$$out.lora_up.weight', 'transformer$$layers$$0$$attention$$qkv.lora_down.weight', 'transformer$$layers$$0$$attention$$qkv.lora_up.weight', 'transformer$$layers$$1$$attention$$out.lora_down.weight', 'transformer$$layers$$1$$attention$$out.lora_up.weight', 'transformer$$layers$$1$$attention$$qkv.lora_down.weight', 'transformer$$layers$$1$$attention$$qkv.lora_up.weight', 'transformer$$layers$$10$$attention$$out.lora_down.weight', 'transformer$$layers$$10$$attention$$out.lora_up.weight', 'transformer$$layers$$10$$attention$$qkv.lora_down.weight', 'transformer$$layers$$10$$attention$$qkv.lora_up.weight', 'transformer$$layers$$11$$attention$$out.lora_down.weight', 'transformer$$layers$$11$$attention$$out.lora_up.weight', 'transformer$$layers$$11$$attention$$qkv.lora_down.weight', 'transformer$$layers$$11$$attention$$qkv.lora_up.weight', 'transformer$$layers$$12$$attention$$out.lora_down.weight', 'transformer$$layers$$12$$attention$$out.lora_up.weight', 'transformer$$layers$$12$$attention$$qkv.lora_down.weight', 'transformer$$layers$$12$$attention$$qkv.lora_up.weight', 'transformer$$layers$$13$$attention$$out.lora_down.weight', 'transformer$$layers$$13$$attention$$out.lora_up.weight', 'transformer$$layers$$13$$attention$$qkv.lora_down.weight', 'transformer$$layers$$13$$attention$$qkv.lora_up.weight', 'transformer$$layers$$14$$attention$$out.lora_down.weight', 'transformer$$layers$$14$$attention$$out.lora_up.weight', 'transformer$$layers$$14$$attention$$qkv.lora_down.weight', 'transformer$$layers$$14$$attention$$qkv.lora_up.weight', 'transformer$$layers$$15$$attention$$out.lora_down.weight', 'transformer$$layers$$15$$attention$$out.lora_up.weight', 'transformer$$layers$$15$$attention$$qkv.lora_down.weight', 'transformer$$layers$$15$$attention$$qkv.lora_up.weight', 'transformer$$layers$$16$$attention$$out.lora_down.weight', 'transformer$$layers$$16$$attention$$out.lora_up.weight', 'transformer$$layers$$16$$attention$$qkv.lora_down.weight', 'transformer$$layers$$16$$attention$$qkv.lora_up.weight', 'transformer$$layers$$17$$attention$$out.lora_down.weight', 'transformer$$layers$$17$$attention$$out.lora_up.weight', 'transformer$$layers$$17$$attention$$qkv.lora_down.weight', 'transformer$$layers$$17$$attention$$qkv.lora_up.weight', 'transformer$$layers$$18$$attention$$out.lora_down.weight', 'transformer$$layers$$18$$attention$$out.lora_up.weight', 'transformer$$layers$$18$$attention$$qkv.lora_down.weight', 'transformer$$layers$$18$$attention$$qkv.lora_up.weight', 'transformer$$layers$$19$$attention$$out.lora_down.weight', 'transformer$$layers$$19$$attention$$out.lora_up.weight', 'transformer$$layers$$19$$attention$$qkv.lora_down.weight', 'transformer$$layers$$19$$attention$$qkv.lora_up.weight', 'transformer$$layers$$2$$attention$$out.lora_down.weight', 'transformer$$layers$$2$$attention$$out.lora_up.weight', 'transformer$$layers$$2$$attention$$qkv.lora_down.weight', 'transformer$$layers$$2$$attention$$qkv.lora_up.weight', 'transformer$$layers$$20$$attention$$out.lora_down.weight', 'transformer$$layers$$20$$attention$$out.lora_up.weight', 'transformer$$layers$$20$$attention$$qkv.lora_down.weight', 'transformer$$layers$$20$$attention$$qkv.lora_up.weight', 'transformer$$layers$$21$$attention$$out.lora_down.weight', 'transformer$$layers$$21$$attention$$out.lora_up.weight', 'transformer$$layers$$21$$attention$$qkv.lora_down.weight', 'transformer$$layers$$21$$attention$$qkv.lora_up.weight', 'transformer$$layers$$22$$attention$$out.lora_down.weight', 'transformer$$layers$$22$$attention$$out.lora_up.weight', 'transformer$$layers$$22$$attention$$qkv.lora_down.weight', 'transformer$$layers$$22$$attention$$qkv.lora_up.weight', 'transformer$$layers$$23$$attention$$out.lora_down.weight', 'transformer$$layers$$23$$attention$$out.lora_up.weight', 'transformer$$layers$$23$$attention$$qkv.lora_down.weight', 'transformer$$layers$$23$$attention$$qkv.lora_up.weight', 'transformer$$layers$$24$$attention$$out.lora_down.weight', 'transformer$$layers$$24$$attention$$out.lora_up.weight', 'transformer$$layers$$24$$attention$$qkv.lora_down.weight', 'transformer$$layers$$24$$attention$$qkv.lora_up.weight', 'transformer$$layers$$25$$attention$$out.lora_down.weight', 'transformer$$layers$$25$$attention$$out.lora_up.weight', 'transformer$$layers$$25$$attention$$qkv.lora_down.weight', 'transformer$$layers$$25$$attention$$qkv.lora_up.weight', 'transformer$$layers$$26$$attention$$out.lora_down.weight', 'transformer$$layers$$26$$attention$$out.lora_up.weight', 'transformer$$layers$$26$$attention$$qkv.lora_down.weight', 'transformer$$layers$$26$$attention$$qkv.lora_up.weight', 'transformer$$layers$$27$$attention$$out.lora_down.weight', 'transformer$$layers$$27$$attention$$out.lora_up.weight', 'transformer$$layers$$27$$attention$$qkv.lora_down.weight', 'transformer$$layers$$27$$attention$$qkv.lora_up.weight', 'transformer$$layers$$28$$attention$$out.lora_down.weight', 'transformer$$layers$$28$$attention$$out.lora_up.weight', 'transformer$$layers$$28$$attention$$qkv.lora_down.weight', 'transformer$$layers$$28$$attention$$qkv.lora_up.weight', 'transformer$$layers$$29$$attention$$out.lora_down.weight', 'transformer$$layers$$29$$attention$$out.lora_up.weight', 'transformer$$layers$$29$$attention$$qkv.lora_down.weight', 'transformer$$layers$$29$$attention$$qkv.lora_up.weight', 'transformer$$layers$$3$$attention$$out.lora_down.weight', 'transformer$$layers$$3$$attention$$out.lora_up.weight', 'transformer$$layers$$3$$attention$$qkv.lora_down.weight', 'transformer$$layers$$3$$attention$$qkv.lora_up.weight', 'transformer$$layers$$4$$attention$$out.lora_down.weight', 'transformer$$layers$$4$$attention$$out.lora_up.weight', 'transformer$$layers$$4$$attention$$qkv.lora_down.weight', 'transformer$$layers$$4$$attention$$qkv.lora_up.weight', 'transformer$$layers$$5$$attention$$out.lora_down.weight', 'transformer$$layers$$5$$attention$$out.lora_up.weight', 'transformer$$layers$$5$$attention$$qkv.lora_down.weight', 'transformer$$layers$$5$$attention$$qkv.lora_up.weight', 'transformer$$layers$$6$$attention$$out.lora_down.weight', 'transformer$$layers$$6$$attention$$out.lora_up.weight', 'transformer$$layers$$6$$attention$$qkv.lora_down.weight', 'transformer$$layers$$6$$attention$$qkv.lora_up.weight', 'transformer$$layers$$7$$attention$$out.lora_down.weight', 'transformer$$layers$$7$$attention$$out.lora_up.weight', 'transformer$$layers$$7$$attention$$qkv.lora_down.weight', 'transformer$$layers$$7$$attention$$qkv.lora_up.weight', 'transformer$$layers$$8$$attention$$out.lora_down.weight', 'transformer$$layers$$8$$attention$$out.lora_up.weight', 'transformer$$layers$$8$$attention$$qkv.lora_down.weight', 'transformer$$layers$$8$$attention$$qkv.lora_up.weight', 'transformer$$layers$$9$$attention$$out.lora_down.weight', 'transformer$$layers$$9$$attention$$out.lora_up.weight', 'transformer$$layers$$9$$attention$$qkv.lora_down.weight', 'transformer$$layers$$9$$attention$$qkv.lora_up.weight', 'transformer$$noise_refiner$$0$$attention$$out.lora_down.weight', 'transformer$$noise_refiner$$0$$attention$$out.lora_up.weight', 'transformer$$noise_refiner$$0$$attention$$qkv.lora_down.weight', 'transformer$$noise_refiner$$0$$attention$$qkv.lora_up.weight', 'transformer$$noise_refiner$$1$$attention$$out.lora_down.weight', 'transformer$$noise_refiner$$1$$attention$$out.lora_up.weight', 'transformer$$noise_refiner$$1$$attention$$qkv.lora_down.weight', 'transformer$$noise_refiner$$1$$attention$$qkv.lora_up.weight', 'transformer$$x_embedder.lora_down.weight', 'transformer$$x_embedder.lora_up.weight', 'transformer$$x_pad_token.lora_down.weight', 'transformer$$x_pad_token.lora_up.weight']
Quantizing Transformer
- quantizing 30 transformer blocks
在aitoolkit中训练，9000步人物还是模糊的，loss还是在0.4左右，0.0002的学习率