Updates to minimal quantization script.

quant_sdxl/README.md

@@ -0,0 +1,33 @@
+# Quantizing SDXL for MLPerf
+
+The purpose of this document is to allow other users to quantize SDXL in various different ways, corresponding to AMD's 2024-07 MLPerf submission.
+
+## Environmental Setup
+
+The environment can be set up as follows:
+
+```bash
+conda env create -n quant_sdxl -f env/brv_sdxl_mlperf_minimal.yml # Use env/brv_sdxl_mlperf_rocm_minimal.yml is using a ROCm compatible GPU
+conda activate quant_sdxl
+```
+
+## Fetching and Pre-Processing Datasets
+
+Information on fetching and pre-processing the data can be found [here](https://github.com/mlcommons/inference/tree/master/text_to_image).
+Note, the latents generated here will be used in the next step.
+
+## Quantize Int8 / FP16 Model
+
+```bash
+python quant_sdxl.py --model stabilityai/stable-diffusion-xl-base-1.0 --device <device> --calibration-prompt-path ./captions.tsv --checkpoint-name unet.ckpt  --path-to-latents <path/to/latents/latents.pt> --guidance-scale 7.5 --exclude-blacklist-act-eq [--path-to-coco <path/to/coco> --validation-prompts 5000]
+```
+
+Note, you can optionally validate on the MLPerf validation set, but be aware this will take ~48hrs.
+
+## Quantize Int8 / FP8 Model
+
+The Int8 / FP8 model can be quantized as follows:
+
+```bash
+python quant_sdxl.py --model stabilityai/stable-diffusion-xl-base-1.0 --device <device> --calibration-prompt-path ./captions.tsv --checkpoint-name unet.ckpt  --path-to-latents <path/to/latents/latents.pt> --guidance-scale 7.5 --quantize-sdp --exclude-blacklist-act-eq [--path-to-coco <path/to/coco> --validation-prompts 5000]
+```

quant_sdxl/captions.tsv

@@ -0,0 +1,501 @@
+id	image_id	caption	height	width	file_name
+282	8965	The cow is rushing towards a moving car.	375	500	COCO_train2014_000000008965.jpg
+655	260395	The bus is traveling down a two way street.	480	640	COCO_train2014_000000260395.jpg
+1919	548183	An overside load on a truck is driving down the road.	480	640	COCO_train2014_000000548183.jpg
+2449	240101	A man in his living room with a game control in his hand.	640	463	COCO_train2014_000000240101.jpg
+3994	362038	Picture of a woman hanging over a toilet.	640	433	COCO_train2014_000000362038.jpg
+4219	480014	two motorcycles and a car parked on the side of an empty road.	640	480	COCO_train2014_000000480014.jpg
+5933	332215	A giraffe standing taller than the trees that surround it	640	428	COCO_train2014_000000332215.jpg
+7631	149233	Baseball players and coach during a practice session.	426	640	COCO_train2014_000000149233.jpg
+8165	398305	A young woman sitting on a bunk bed.	640	463	COCO_train2014_000000398305.jpg
+8560	379743	A sign warning drivers to slow down because of the presence of children.	431	640	COCO_train2014_000000379743.jpg
+11546	357443	A gold clock featuring a man and a bird.	427	640	COCO_train2014_000000357443.jpg
+12209	408294	A woman stands up to play a game while others watch.	334	500	COCO_train2014_000000408294.jpg
+12214	207075	A fighter jet flying through a blue cloud filled sky.	427	640	COCO_train2014_000000207075.jpg
+12310	119938	Two zebras standing together staring at other animals in the distance	427	640	COCO_train2014_000000119938.jpg
+12824	132646	A cat sitting next to a sports water bottle.	427	640	COCO_train2014_000000132646.jpg
+16062	43210	A chocolate cake with candles that is shaped like a six-legged creature.	458	640	COCO_train2014_000000043210.jpg
+16859	334048	A giraffe on the plains with a mountain in the background.	640	492	COCO_train2014_000000334048.jpg
+16963	387295	A calico cat sitting underneath a metal bench,	428	640	COCO_train2014_000000387295.jpg
+19857	240321	A man is sitting on the ground looking at his phone.	427	640	COCO_train2014_000000240321.jpg
+20118	139124	A old clock and sculptures of people in blue outfits.	438	640	COCO_train2014_000000139124.jpg
+22609	232186	A bowl of pasta with meat and broccoli.	425	640	COCO_train2014_000000232186.jpg
+22759	68252	A baseball player attempts a slide as a catcher and umpire look on.	426	640	COCO_train2014_000000068252.jpg
+25891	159403	close up of toy truck on a table	511	640	COCO_train2014_000000159403.jpg
+34997	164310	A person water skiing on the ocean from a boat.	481	640	COCO_train2014_000000164310.jpg
+35274	205356	A restroom with a toilet next to a sink.	640	480	COCO_train2014_000000205356.jpg
+37691	289852	A plate with the remains of a dessert and congratulations written on it with three spoons.	480	640	COCO_train2014_000000289852.jpg
+38242	150562	a white swan is floating in the water	360	640	COCO_train2014_000000150562.jpg
+38276	353610	A man and a woman standing under an open umbrella.	429	640	COCO_train2014_000000353610.jpg
+43640	277857	An Asian woman is eating a meal in a restaurant.	429	640	COCO_train2014_000000277857.jpg
+43812	316093	A group of people ride horses up a hill	480	640	COCO_train2014_000000316093.jpg
+46608	395170	A black cat is sitting in a black suitcase	480	640	COCO_train2014_000000395170.jpg
+51634	486451	a silver gray subway train parked in a subway	361	640	COCO_train2014_000000486451.jpg
+54503	425742	Three women all looking at something on a laptop computer screen.	480	640	COCO_train2014_000000425742.jpg
+57560	69654	a train sitting next to  a  pedestrian sitting on a bench on a  railway platform.	471	640	COCO_train2014_000000069654.jpg
+61381	189304	A white-haired bearded man tosses a ball underhand from the pitcher's mound.	640	427	COCO_train2014_000000189304.jpg
+61482	188787	a baseball player swinging a bat at a ball	426	640	COCO_train2014_000000188787.jpg
+64888	411703	A plate with a sandwich and green pepper on a tray with a salad in a clear bowl and clear glass of water.	480	640	COCO_train2014_000000411703.jpg
+68709	415912	A man holding on to bars while riding a water ski	640	640	COCO_train2014_000000415912.jpg
+72435	318521	A man standing on a white toilet in a restroom stall.	500	333	COCO_train2014_000000318521.jpg
+74694	1098	A woman kneeling down to pet a black and white cow laying on the ground.	532	640	COCO_train2014_000000001098.jpg
+75279	119187	A plate filled with food sitting on top of a table.	612	612	COCO_train2014_000000119187.jpg
+75409	368873	Two adult Zebra grazing in a natural setting.	480	640	COCO_train2014_000000368873.jpg
+75664	276703	A POLICE CAR PARKED IN THE STREET WITH TREES BEHIND IT.	375	500	COCO_train2014_000000276703.jpg
+76297	313648	A man wearing headphones while chewing on an ink pen.	640	480	COCO_train2014_000000313648.jpg
+76700	335308	A colorful Christmas tree sitting next to couches in a living room.	327	640	COCO_train2014_000000335308.jpg
+77959	509870	A desktop computer and a laptop are running at a desk overlooking a meeting area.	375	500	COCO_train2014_000000509870.jpg
+78032	561097	Black and white picture of two horses standing in a field.	428	640	COCO_train2014_000000561097.jpg
+79803	337262	A guy skateboarding down a street with two bicyclists not far behind amongst palm trees.	428	640	COCO_train2014_000000337262.jpg
+82363	573922	A series of different airplanes going through the sky.	480	640	COCO_train2014_000000573922.jpg
+84527	247459	A green traffic signal next to a large green sign.	640	428	COCO_train2014_000000247459.jpg
+90667	193594	A chef is mixing ingredients in a silver bowl	480	640	COCO_train2014_000000193594.jpg
+91685	449508	An orange and blue passenger boat floats in the canal, making it's way past other boats that are docked there.	427	640	COCO_train2014_000000449508.jpg
+96724	433397	Geeks on Call worker holds an Apple MacBook.	640	449	COCO_train2014_000000433397.jpg
+98481	303227	A donkey walking down a street while others walk along it.	437	640	COCO_train2014_000000303227.jpg
+99060	308366	an orange bowl with a brown roll in it	343	480	COCO_train2014_000000308366.jpg
+99425	325524	a traffic sign with a camera behind a empty street	640	480	COCO_train2014_000000325524.jpg
+100234	301778	Dirty bathroom floor with a toilet and a toilet brush next to it.	640	427	COCO_train2014_000000301778.jpg
+100859	221045	a bowl of cereal with bananas, and a drink	334	500	COCO_train2014_000000221045.jpg
+106223	242467	A square cut pizza sitting on top of a table with a spatula.	480	640	COCO_train2014_000000242467.jpg
+106965	151885	A man bending over to pick up a colorful kite.	480	640	COCO_train2014_000000151885.jpg
+109094	219527	Three stuffed bears laying on a bed next to a stuffed frog.	480	640	COCO_train2014_000000219527.jpg
+109822	190043	A yellow fire hydrant sitting in front of a blue brick building.	333	500	COCO_train2014_000000190043.jpg
+110271	262273	A man standing on skis while holding two ski poles.	640	480	COCO_train2014_000000262273.jpg
+111258	544832	A woman standing in a room watching television.	480	640	COCO_train2014_000000544832.jpg
+113365	479301	A sandwich and greens sit together on a plate.	478	640	COCO_train2014_000000479301.jpg
+116179	229193	A man is standing up slicing a pizza on the counter	640	429	COCO_train2014_000000229193.jpg
+118646	168454	A black and white photo of some food in a bowl.	426	640	COCO_train2014_000000168454.jpg
+119116	176481	A male cooking onions on a stove.	518	387	COCO_train2014_000000176481.jpg
+120604	487876	A young man doing skateboard tricks at the skate park.	428	640	COCO_train2014_000000487876.jpg
+122972	89196	a close up of food on a plate with meat and broccoli	427	640	COCO_train2014_000000089196.jpg
+123104	117585	a black and white photo of a person sitting on a motorcycle	640	398	COCO_train2014_000000117585.jpg
+123235	295328	A street scene with a woman in a crosswalk and a taxi.	622	640	COCO_train2014_000000295328.jpg
+123407	326236	A group of men racing horses on top of a beach.	442	640	COCO_train2014_000000326236.jpg
+127408	55413	a man and women sitting on a motorcycle with an American flag on the back	480	640	COCO_train2014_000000055413.jpg
+129239	28759	Some people picking up luggage at the airport.	480	640	COCO_train2014_000000028759.jpg
+130755	62053	A cluttered desk with electronics desks and papers	480	640	COCO_train2014_000000062053.jpg
+132461	59921	A man looking at a truck parked near a barn	425	640	COCO_train2014_000000059921.jpg
+132928	182367	A man using a blender to make a drink as a crowd looks on.	428	640	COCO_train2014_000000182367.jpg
+135203	354993	A picture of a person and some luggage on the ground.	480	640	COCO_train2014_000000354993.jpg
+135647	61409	A car that is on the side of the road.	426	640	COCO_train2014_000000061409.jpg
+135956	552403	A piece of food and a drink on a table.	375	500	COCO_train2014_000000552403.jpg
+137747	93075	The person is skiing in the snowy area.	426	640	COCO_train2014_000000093075.jpg
+137832	7116	A couple of small boats are on the lake.	427	640	COCO_train2014_000000007116.jpg
+139916	185832	a person standing in the living room with other people sitting on couches	640	480	COCO_train2014_000000185832.jpg
+145736	132576	A man riding skis off the side of a ramp in the snow.	360	640	COCO_train2014_000000132576.jpg
+147243	555998	A table topped with the contents of a woman's purse.	480	640	COCO_train2014_000000555998.jpg
+147892	233925	A gas sign post on a dark street.	640	480	COCO_train2014_000000233925.jpg
+153816	512068	A large cat sitting under a sign that reads no ball games.	500	451	COCO_train2014_000000512068.jpg
+155022	90328	A living room filled with furniture and large windows over looking a city.	487	600	COCO_train2014_000000090328.jpg
+155047	496496	A beautiful woman n a bikini standing on a wet beach.	428	640	COCO_train2014_000000496496.jpg
+158009	263727	A clock tower with a variety of lights on top of it.	640	480	COCO_train2014_000000263727.jpg
+158557	542213	A man eating out of a box in a kitchen.	480	640	COCO_train2014_000000542213.jpg
+159299	347555	Lady wearing a black tie with green and lavender eye shadow.	426	300	COCO_train2014_000000347555.jpg
+164206	440984	A beautiful young woman holding a tennis racquet on a court.	426	640	COCO_train2014_000000440984.jpg
+165028	130473	A black bear sitting on top of a fallen log at a zoo.	425	640	COCO_train2014_000000130473.jpg
+165957	128608	A woman in a home kitchen with vertical blinds on the windows	360	640	COCO_train2014_000000128608.jpg
+165985	384122	a coach bus going under a colorful archway	422	640	COCO_train2014_000000384122.jpg
+167864	521184	A man leads a horse by the reigns while a woman looks on.	427	640	COCO_train2014_000000521184.jpg
+170830	352841	A kitchen scene with focus on a kitchen cabinet.	640	480	COCO_train2014_000000352841.jpg
+171117	218941	A baby sitting in a white high chair eating a piece of cake.	425	640	COCO_train2014_000000218941.jpg
+176908	315242	A boy is holding up a snowboard while sitting in the snow outside a lodge.	333	500	COCO_train2014_000000315242.jpg
+180042	109208	A woman standing on a tennis court with a racquet.	480	640	COCO_train2014_000000109208.jpg
+185826	209400	Horses and a foal eating grass in a pasture	386	640	COCO_train2014_000000209400.jpg
+185932	255010	Black and white photo of two men with black hats sitting in a train.	331	500	COCO_train2014_000000255010.jpg
+189915	509176	A plate topped with fruit pastries stacked on top of each other.	333	500	COCO_train2014_000000509176.jpg
+194228	84873	A square pizza sitting on top of a table.	480	640	COCO_train2014_000000084873.jpg
+195160	132249	THERE IS AN ELEPHANT STANDING IN THE WATER	500	333	COCO_train2014_000000132249.jpg
+195713	202752	A white refrigerator next to a counter with an orange box.	640	480	COCO_train2014_000000202752.jpg
+197848	460118	A group of sheep grazing in a canyon.	428	640	COCO_train2014_000000460118.jpg
+199178	29730	A woman riding skis down a snow covered slope.	429	640	COCO_train2014_000000029730.jpg
+199200	310833	A bathroom with tiled floor and a white sink.	640	480	COCO_train2014_000000310833.jpg
+199908	157696	A young man wearing a catchers mitt standing on a grass field.	428	640	COCO_train2014_000000157696.jpg
+199921	508100	A group of sheep are grazing in the grass near a mountain.	428	640	COCO_train2014_000000508100.jpg
+202501	219784	A man in a black hat and orange hoodie eating a slice of pizza.	480	640	COCO_train2014_000000219784.jpg
+204193	188511	A bathroom with a white toilet next to a large window.	640	480	COCO_train2014_000000188511.jpg
+209966	132183	An animal sticking out his head from a fence eating.	427	640	COCO_train2014_000000132183.jpg
+214011	363719	Two men at a wooden table with a pizza.	375	500	COCO_train2014_000000363719.jpg
+214229	35400	A giraffe laying in a lush green grass covered field.	426	640	COCO_train2014_000000035400.jpg
+214586	188454	A small bird perched on top of a tree branch.	428	640	COCO_train2014_000000188454.jpg
+214867	517494	Two zebras standing in water with trees in the background.	359	640	COCO_train2014_000000517494.jpg
+217065	475696	A woman in lingerie wearing black high heels.	427	640	COCO_train2014_000000475696.jpg
+217295	69614	A silver passenger train parked at a train station	640	480	COCO_train2014_000000069614.jpg
+217484	579659	A dining room filled with tables and chairs and lighting.	427	640	COCO_train2014_000000579659.jpg
+220690	527056	A red table topped with pants and plates of food.	612	612	COCO_train2014_000000527056.jpg
+220722	371382	A crowded street filled with lots of traffic.	640	366	COCO_train2014_000000371382.jpg
+223208	52956	A bowl filled with ice cream and berries next to a plate of food.	320	640	COCO_train2014_000000052956.jpg
+225982	222654	A small dog walking across a beach next to a forest.	480	640	COCO_train2014_000000222654.jpg
+226008	571745	A white toilet in a black restroom stall.	640	480	COCO_train2014_000000571745.jpg
+228286	58708	A woman wearing a white shirt and red tie.	640	480	COCO_train2014_000000058708.jpg
+229410	179154	Various cars passing an intersection under traffic lights.	480	640	COCO_train2014_000000179154.jpg
+231454	319216	A bird that is standing on a beach during sunset.	640	480	COCO_train2014_000000319216.jpg
+231675	562675	a close up of heart heart shaped cakes	427	640	COCO_train2014_000000562675.jpg
+232564	450212	An old vintage truck sitting in a parking lot.	440	640	COCO_train2014_000000450212.jpg
+233822	178410	Children with the help of theirir fathers playing video games	427	640	COCO_train2014_000000178410.jpg
+238605	13195	The man is hitting the ball during a baseball game.	500	491	COCO_train2014_000000013195.jpg
+238703	440283	Two clocks on a tower near a building with american flags.	640	480	COCO_train2014_000000440283.jpg
+239720	102080	an image of a small elephant and a large one	640	549	COCO_train2014_000000102080.jpg
+241728	522481	Ten small round dishes containing various food items.	480	640	COCO_train2014_000000522481.jpg
+242835	168690	A laptop and keyboard are sitting on a white desk in a hotel room.	480	640	COCO_train2014_000000168690.jpg
+248687	311377	A skateboarder grinds his skateboard on a metal railing.	424	640	COCO_train2014_000000311377.jpg
+254060	403999	A woman playing a video game with a remote controller.	480	640	COCO_train2014_000000403999.jpg
+255054	273421	A bathroom filled with a bath tub next to a sink under a window.	400	640	COCO_train2014_000000273421.jpg
+257247	279197	A man on a surf board riding a wave in the ocean.	427	640	COCO_train2014_000000279197.jpg
+260915	198607	A woman is hugging her small child on the couch.	480	640	COCO_train2014_000000198607.jpg
+262962	468420	A group of people standing around orange and white cones.	426	640	COCO_train2014_000000468420.jpg
+263381	122093	A baseball player holding a baseball bat on a field.	428	640	COCO_train2014_000000122093.jpg
+264082	500136	A brown table sitting in a room next to a lot of books.	480	640	COCO_train2014_000000500136.jpg
+266208	554642	A red brick building next to a green door.	500	333	COCO_train2014_000000554642.jpg
+266360	425542	A pretty young female tennis player holding a racket.	429	640	COCO_train2014_000000425542.jpg
+270221	120697	A man getting ready to bat while people watch from behind him.	333	500	COCO_train2014_000000120697.jpg
+271934	93078	Several people standing around a table full of beer bottles.	500	375	COCO_train2014_000000093078.jpg
+274449	292432	Two passenger trains sits on the edges of a platform.	480	640	COCO_train2014_000000292432.jpg
+277740	367934	an image of a man that is on floor playing with child	427	640	COCO_train2014_000000367934.jpg
+278168	23361	Two cars sit at an intersection with crosswalks.	640	480	COCO_train2014_000000023361.jpg
+279180	162801	a cat sitting on a floor near some pairs of shoes	424	640	COCO_train2014_000000162801.jpg
+284113	29541	a close up of a person wearing a shirt and tie	480	640	COCO_train2014_000000029541.jpg
+284902	283728	A man holding a kite on top of a lush green field.	347	640	COCO_train2014_000000283728.jpg
+285577	142420	A refrigerator filled with jars of food and juice.	480	640	COCO_train2014_000000142420.jpg
+286143	346360	Two men standing in a living room by a couch playing wii.	480	640	COCO_train2014_000000346360.jpg
+287011	164036	A traffic light on the corner of a city street.	640	640	COCO_train2014_000000164036.jpg
+288123	353694	A truck with a bus in tow behind.	480	640	COCO_train2014_000000353694.jpg
+288148	426663	a man dressed like a cupcake next to a horse	480	640	COCO_train2014_000000426663.jpg
+288191	151371	A parked truck and car facing each other on a street.	427	640	COCO_train2014_000000151371.jpg
+289544	296581	A very sleek and modern apartment decorated mostly in white.	426	640	COCO_train2014_000000296581.jpg
+289820	252139	Animals roam on the hills where the water streaming down the side of it.	360	640	COCO_train2014_000000252139.jpg
+289895	188727	Two giraffe laying in a open grassy field.	480	640	COCO_train2014_000000188727.jpg
+297749	422520	A cat sits on the rungs of a chair that's hung up.	640	480	COCO_train2014_000000422520.jpg
+299866	166993	A wooden bench sitting next to a lake.	438	640	COCO_train2014_000000166993.jpg
+302385	203809	A person standing on a sidewalk on a cell phone.	500	332	COCO_train2014_000000203809.jpg
+306804	540633	A glass cake plate with a double layer cake on top of it.	428	640	COCO_train2014_000000540633.jpg
+307451	170588	A large building and  tower with a clock.	424	640	COCO_train2014_000000170588.jpg
+308370	289071	A directional sign pointing to Capitol, Lake Monona and other destinations.	640	480	COCO_train2014_000000289071.jpg
+311176	201236	A male tennis player in mid air action on the court.	640	426	COCO_train2014_000000201236.jpg
+311859	149422	A line of white toilets sit against a brown wall.	480	640	COCO_train2014_000000149422.jpg
+312484	21740	A kitchen area with table and bar stools with a window view of the lake.	480	640	COCO_train2014_000000021740.jpg
+313043	552852	These boys are sitting at a table behind a bunch of bananas.	427	640	COCO_train2014_000000552852.jpg
+315973	568788	A manipulated photo of a dog in a mirror with and without a collar.	480	640	COCO_train2014_000000568788.jpg
+317878	340102	A woman kneeling on the floor using a food processor.	472	640	COCO_train2014_000000340102.jpg
+321959	211381	a subway sitting on the tracks at the station	427	640	COCO_train2014_000000211381.jpg
+322372	135966	A man walking down a street holding a black umbrella.	480	640	COCO_train2014_000000135966.jpg
+324354	64036	A group of men playing soccer on a grass field.	493	640	COCO_train2014_000000064036.jpg
+324846	397525	A girl in yellow and red shirt sitting next to a cake.	640	480	COCO_train2014_000000397525.jpg
+326131	7143	An airplane is flying over nice blue water.	500	281	COCO_train2014_000000007143.jpg
+327858	240689	a young child on a beach with a kite flying in the sky	425	640	COCO_train2014_000000240689.jpg
+328638	227178	A chair and a table next to a kitchen.	480	640	COCO_train2014_000000227178.jpg
+330754	260175	A train stopped at a train station with its doors still closed..	427	640	COCO_train2014_000000260175.jpg
+331208	109774	A man posing next to a fence by an enclosure with a herd of zebras at a zoo.	427	640	COCO_train2014_000000109774.jpg
+331351	107176	A table with laptop on it with an extra hard drive hooked to it.	427	640	COCO_train2014_000000107176.jpg
+331947	55651	two people in a kitchen area with sinks and a stove	426	640	COCO_train2014_000000055651.jpg
+332345	172889	A picture of a toilet inside of a room.	480	640	COCO_train2014_000000172889.jpg
+332615	475438	A group of people skiing on the mountain with snow	375	500	COCO_train2014_000000475438.jpg
+332621	116402	A family playing video games in front of the television.	480	640	COCO_train2014_000000116402.jpg
+339395	445893	a man in a had drives a vintage fire truck	480	640	COCO_train2014_000000445893.jpg
+344215	450584	A vase of white flowers and greenery on a table.	428	640	COCO_train2014_000000450584.jpg
+345425	236218	a corner of a small bathroom with a sink next to a window	640	479	COCO_train2014_000000236218.jpg
+345429	319354	The back end of a parked car with an umbrella on the ground next to it.	480	640	COCO_train2014_000000319354.jpg
+345857	139120	A man choosing a piece of pizza from two boxes	427	640	COCO_train2014_000000139120.jpg
+346481	68662	There are some people paddling kayaks in the river	480	640	COCO_train2014_000000068662.jpg
+348287	289005	a bunch of motorcycles sit lined up with each other	640	427	COCO_train2014_000000289005.jpg
+351866	486062	A black and grey bird is walking across a street.	640	480	COCO_train2014_000000486062.jpg
+352586	325681	a person sending a text on her phone	287	640	COCO_train2014_000000325681.jpg
+353202	561442	Cars ride on a suburban street with trees.	500	375	COCO_train2014_000000561442.jpg
+353718	306128	A pair of men playing against each other in frisbee.	428	640	COCO_train2014_000000306128.jpg
+355844	325021	A man and a woman standing over a table filled with food.	427	640	COCO_train2014_000000325021.jpg
+365667	492610	A train taking a bridge over a frozen river.	480	640	COCO_train2014_000000492610.jpg
+367550	377322	A girl and boy playing with a WII at a get together.	480	640	COCO_train2014_000000377322.jpg
+371196	407693	A man holds a clock at his torso in an art room.	361	640	COCO_train2014_000000407693.jpg
+371926	511357	three people sitting in chairs and a teddy bear	356	500	COCO_train2014_000000511357.jpg
+372654	29964	A woman takes a swing at a tennis ball that is high in the air.	640	403	COCO_train2014_000000029964.jpg
+373049	44672	A man wearing skiis standing in the snow.	427	640	COCO_train2014_000000044672.jpg
+374328	270323	A clock tower on top of a building with a wind indicator.	640	512	COCO_train2014_000000270323.jpg
+375130	39040	A group of people with umbrellas stand in the road.	360	640	COCO_train2014_000000039040.jpg
+378239	362722	A white bus turning on street next to a building.	469	640	COCO_train2014_000000362722.jpg
+380251	134799	A person holding up their phone to record something	424	640	COCO_train2014_000000134799.jpg
+380941	228664	a young child swinging a bat at a ball	375	500	COCO_train2014_000000228664.jpg
+391654	531201	The man is playing a game of tennis on the court.	480	640	COCO_train2014_000000531201.jpg
+393623	349588	A large clock tower in front of a tall sky scraper	333	500	COCO_train2014_000000349588.jpg
+394781	466294	A traffic signal is on the street beside a building.	357	500	COCO_train2014_000000466294.jpg
+398013	532014	A man is watching through a fence as a boy plays baseball.	427	640	COCO_train2014_000000532014.jpg
+401355	311087	A bunch of vegetables are on the table by the sink.	478	640	COCO_train2014_000000311087.jpg
+406364	532188	a woman and her kids standing at a carnival	640	480	COCO_train2014_000000532188.jpg
+407134	486960	a close up of two zebras near one another near trees	481	500	COCO_train2014_000000486960.jpg
+409035	377756	A black car parked near an intersection with a no turn sign	480	640	COCO_train2014_000000377756.jpg
+409580	372193	A man riding on his surfboard in the ocean waves.	427	640	COCO_train2014_000000372193.jpg
+411447	194050	A man jumping in the air on a snowboard.	427	640	COCO_train2014_000000194050.jpg
+412219	253686	a girl with an umbrella in the rain	640	427	COCO_train2014_000000253686.jpg
+413745	17105	An empty room with various pieces of colourful furniture.	426	640	COCO_train2014_000000017105.jpg
+414198	178987	Up close view of a plate with two well cooked hot dogs on it.	427	640	COCO_train2014_000000178987.jpg
+414247	92089	A hand holding two hot dogs on napkins.	427	640	COCO_train2014_000000092089.jpg
+419296	524773	a stop sign sits at a corner of a street	480	640	COCO_train2014_000000524773.jpg
+419958	468706	Motorcyclist riding in shallow water at ocean beach.	333	500	COCO_train2014_000000468706.jpg
+420531	99417	A chocolate birthday cake decorate with bicycles on foil	480	640	COCO_train2014_000000099417.jpg
+420930	549236	Two dogs are watching a television set intently.	480	640	COCO_train2014_000000549236.jpg
+421176	6590	A young child on a sofa holding an umbrella	480	640	COCO_train2014_000000006590.jpg
+421294	423052	A sign and car on a city street.	500	375	COCO_train2014_000000423052.jpg
+422126	378848	A bicycle that is being slowly covered over by some vines.	480	640	COCO_train2014_000000378848.jpg
+422385	7090	The pizza for lunch are on the trays.	480	640	COCO_train2014_000000007090.jpg
+424452	232770	A boy handing a man an orange by a white board.	640	640	COCO_train2014_000000232770.jpg
+424861	286703	A pair of shoe sit underneath a bed.	423	640	COCO_train2014_000000286703.jpg
+425835	121965	two cats, one orange and one gray, sit on  a shelf intended for shoes	480	640	COCO_train2014_000000121965.jpg
+430083	509419	A woman is bent over looking into an aquarium	480	640	COCO_train2014_000000509419.jpg
+430684	476736	A dog standing on a skateboard on the sidewalk.	426	640	COCO_train2014_000000476736.jpg
+432654	75267	A large white airplane on a open road.	426	640	COCO_train2014_000000075267.jpg
+433701	556279	A man in a graduation outfit posing with a lady.	640	427	COCO_train2014_000000556279.jpg
+433886	268306	An old  classic motorcycle on display inside a big museum.	640	426	COCO_train2014_000000268306.jpg
+435192	147890	A dog jumps into the water and kicks up some mud or sand.	558	640	COCO_train2014_000000147890.jpg
+436632	32455	A city street filled with traffic next to a mountain.	333	500	COCO_train2014_000000032455.jpg
+437000	153019	Two tennis players are holding rackets and looking at a cell phone.	500	372	COCO_train2014_000000153019.jpg
+437118	250195	a person walking on a city street near a traffic light	640	640	COCO_train2014_000000250195.jpg
+438972	244365	A white and black horse pulling a cart in a street.	640	390	COCO_train2014_000000244365.jpg
+439208	456217	Two men and a women in the middle wearing a floral top	480	640	COCO_train2014_000000456217.jpg
+439856	146705	Small public toilet with a black toilet paper dispenser next to it.	425	640	COCO_train2014_000000146705.jpg
+440473	68797	Many children playing soccer in an outdoor field.	427	640	COCO_train2014_000000068797.jpg
+442129	206583	people sitting around a restaurant table with a bright table cloth	426	640	COCO_train2014_000000206583.jpg
+443682	192627	A group of people flying kites at the beach.	424	640	COCO_train2014_000000192627.jpg
+445552	24712	two men pose for a picture while holding rackets	480	640	COCO_train2014_000000024712.jpg
+446573	145699	Two white urinals can be seen as you enter the door with a sign on it.	640	480	COCO_train2014_000000145699.jpg
+446883	360370	A group of black and white cows standing on a lush green field.	425	640	COCO_train2014_000000360370.jpg
+447159	479045	A sidewalk view of shops and a clock.	427	640	COCO_train2014_000000479045.jpg
+449091	445425	Young baseball player hitting ball during game with onlookers.	427	640	COCO_train2014_000000445425.jpg
+449276	307432	a baseball player holding a bat on a field	480	640	COCO_train2014_000000307432.jpg
+452271	443479	A man decoration a plate of food with red sauce.	640	480	COCO_train2014_000000443479.jpg
+452384	372685	Five people holding a knife together and cutting a cake.	425	640	COCO_train2014_000000372685.jpg
+456664	91135	there are many different animals that can be seen in the wild	375	500	COCO_train2014_000000091135.jpg
+457747	439615	A black and white image of many people around motorcycles.	574	640	COCO_train2014_000000439615.jpg
+460835	402915	there is a yellow truck sign next to a fire hydrant	640	480	COCO_train2014_000000402915.jpg
+466636	192307	A woman leaning on a pole holding two traffic signs.	640	429	COCO_train2014_000000192307.jpg
+466793	498295	A young woman sitting at a table eating a delicious meal.	640	435	COCO_train2014_000000498295.jpg
+468725	181356	A man cutting cheese into cubes on a cutting board.	375	500	COCO_train2014_000000181356.jpg
+470690	552368	Villagers sit on the sides of the roadway.	431	640	COCO_train2014_000000552368.jpg
+474368	305348	A male tennis player is on the court.	476	640	COCO_train2014_000000305348.jpg
+475067	535995	A woman standing on a beach holding up a colorful kite.	375	500	COCO_train2014_000000535995.jpg
+476921	374001	A big collection of traffic lights are on display near a street.	422	640	COCO_train2014_000000374001.jpg
+477743	343048	A street sign is shown amongst some buildings.	480	640	COCO_train2014_000000343048.jpg
+479422	131915	Three decorated ponies stand with a lady facing a building.	640	427	COCO_train2014_000000131915.jpg
+482315	466288	People standing by a motorcycle in the mountains	426	640	COCO_train2014_000000466288.jpg
+483783	364559	A person skiing in the snow, down a mountain.	402	604	COCO_train2014_000000364559.jpg
+484960	127388	A woman is playing frisbee while a young man poses next to her.	458	640	COCO_train2014_000000127388.jpg
+485886	7838	A group of people on a park bench under a tree	478	640	COCO_train2014_000000007838.jpg
+487773	154161	A beautiful custom black and blue motorcycle in a parking space.	408	640	COCO_train2014_000000154161.jpg
+488784	420926	A man flying a large parachute like kite	457	640	COCO_train2014_000000420926.jpg
+489599	550955	A man hitting a tennis ball with a racquet.	426	640	COCO_train2014_000000550955.jpg
+493865	209356	Two submarine sandwiches and two bags of chips.	480	640	COCO_train2014_000000209356.jpg
+494659	576994	A living room area with matching couches and a beige carpet.	427	640	COCO_train2014_000000576994.jpg
+495526	491801	a toaster oven with a pan of food inside and a pan of toast on top	480	640	COCO_train2014_000000491801.jpg
+496530	225329	A double decker bus driving next to a  man on a bike.	426	640	COCO_train2014_000000225329.jpg
+497729	523495	there is a man drinking and eating hot dogs	640	480	COCO_train2014_000000523495.jpg
+502146	275913	a fire hydrant buried in tall grass and flowers	640	480	COCO_train2014_000000275913.jpg
+502397	353761	A light rail train is sitting in a lit up train station.	427	640	COCO_train2014_000000353761.jpg
+503045	479399	a bunch of traffic as a cloudy sky hangs over	307	500	COCO_train2014_000000479399.jpg
+503682	75174	two people on a motorcycle on a city street	427	640	COCO_train2014_000000075174.jpg
+504091	202499	A person with a knife near a cutting board on a counter.	283	500	COCO_train2014_000000202499.jpg
+504506	151566	A man with an apron on making a vase.	330	500	COCO_train2014_000000151566.jpg
+504597	107903	a living room with a big tv at one end of it	612	612	COCO_train2014_000000107903.jpg
+505156	39768	A picture of John Kerry hold flowers for a girl.	426	640	COCO_train2014_000000039768.jpg
+505840	274897	A black and white photo of a plant against a wall.	640	414	COCO_train2014_000000274897.jpg
+506000	466964	A market where umbrellas are hung upside down.	480	640	COCO_train2014_000000466964.jpg
+506025	365479	a bathroom with large mirror, a toilet and sink and a bathtub with a shower that has a sliding door	268	640	COCO_train2014_000000365479.jpg
+507740	492853	A large bear walking along a river near trees.	640	563	COCO_train2014_000000492853.jpg
+508059	361519	A rural dirt road sitting between very large snow covered mountains.	332	500	COCO_train2014_000000361519.jpg
+508871	4442	A little girl sitting on top of a bed near pillows.	427	640	COCO_train2014_000000004442.jpg
+509713	339248	A group of men standing in a room next to each other.	480	640	COCO_train2014_000000339248.jpg
+512137	88720	A man is talking to a woman by a car.	348	500	COCO_train2014_000000088720.jpg
+512356	212276	A DOG SPLASHING RUNNING IN WATER PLAYING FRISBEE.	446	640	COCO_train2014_000000212276.jpg
+512959	306284	Three black cows have white spots on them.	427	640	COCO_train2014_000000306284.jpg
+513858	72156	The skateboarder in a tan cap balances on his board.	427	640	COCO_train2014_000000072156.jpg
+514578	207447	The two people are helping the older woman out.	360	640	COCO_train2014_000000207447.jpg
+515325	57445	A dog looking intently at something by a persons laptop.	427	640	COCO_train2014_000000057445.jpg
+515457	201352	A man doing a handstand with one hand on a skateboard	628	640	COCO_train2014_000000201352.jpg
+515641	179580	A green vase pitcher with a bouquet of flowers in it.	500	375	COCO_train2014_000000179580.jpg
+515979	550097	Two boats colliding with each other on a river.	427	640	COCO_train2014_000000550097.jpg
+517609	263302	The bathroom is being token apart for remodel	480	640	COCO_train2014_000000263302.jpg
+517616	354933	A lone skier travels cross country through the woods	360	640	COCO_train2014_000000354933.jpg
+521598	295166	A room with cabinet and rug in it.	480	640	COCO_train2014_000000295166.jpg
+522575	401603	there is a man skateboarding in a  skateboard park.	640	428	COCO_train2014_000000401603.jpg
+522603	572745	A woman tossing a white frisbee into a fountain.	427	640	COCO_train2014_000000572745.jpg
+523504	517087	The man is standing in the open area wearing  a hat.	427	640	COCO_train2014_000000517087.jpg
+523759	23908	A brown and white cow standing on top of a dry grass field.	480	640	COCO_train2014_000000023908.jpg
+524253	78359	A office caddy holding pens, scissors, and a ruler	640	428	COCO_train2014_000000078359.jpg
+525438	323457	A herd of cattle that are grazing in a field.	427	640	COCO_train2014_000000323457.jpg
+526933	327281	A car at a stoplight with its brake lights on.	375	500	COCO_train2014_000000327281.jpg
+527636	514619	A large bus is driving down a street lined with large buildings.	480	640	COCO_train2014_000000514619.jpg
+528067	16720	An American Flag flies atop a clock tower at sports stadium Coors Field.	424	640	COCO_train2014_000000016720.jpg
+531764	53729	A crowd of people standing on top of a sandy beach.	480	640	COCO_train2014_000000053729.jpg
+531912	34454	A woman holding a dog while lying on a surf board.	427	640	COCO_train2014_000000034454.jpg
+532263	228657	A man riding a skateboard up the side of a ramp.	425	640	COCO_train2014_000000228657.jpg
+534126	440894	Herd of Wilde beast and zebra walk through grass by shore line	358	640	COCO_train2014_000000440894.jpg
+534769	166575	A group of zebras grazing in a field.	310	640	COCO_train2014_000000166575.jpg
+536579	324953	a couple of people standing in a kitchen	424	640	COCO_train2014_000000324953.jpg
+539759	534291	A row of white toilets sitting next to each other on the floor.	427	640	COCO_train2014_000000534291.jpg
+540760	167743	Man sitting down and talking on his cell phone.	425	640	COCO_train2014_000000167743.jpg
+540832	543754	Two men holding surfboards while standing in the ocean.	480	640	COCO_train2014_000000543754.jpg
+542911	63949	Two people with surfboards at the beach close to the ocean	480	640	COCO_train2014_000000063949.jpg
+543668	414738	a person wearing Levi jeans is holding a toothbrush	573	640	COCO_train2014_000000414738.jpg
+545231	565939	A man standing in front of  a soccer goal with a dog.	428	640	COCO_train2014_000000565939.jpg
+546508	383390	some kids playing baseball as the parents watch them	480	640	COCO_train2014_000000383390.jpg
+549123	526840	A field full of sheep grazing on lush green grass.	480	640	COCO_train2014_000000526840.jpg
+550299	34415	A cat laying on top of a large cat bed.	332	500	COCO_train2014_000000034415.jpg
+552353	451574	A blue, white and red Air Tahiti Nui airplane is shown.	210	640	COCO_train2014_000000451574.jpg
+557549	97363	a stop sign leaving a shadow on the grass beside it	480	640	COCO_train2014_000000097363.jpg
+559519	481665	The  competition  rider is mounted on the  prancing horse.	640	425	COCO_train2014_000000481665.jpg
+560177	304473	A person on a field throwing a ball.	491	500	COCO_train2014_000000304473.jpg
+563283	180559	The two people are reaching for the Frisbee.	425	640	COCO_train2014_000000180559.jpg
+565069	407441	A person on a court with a tennis racket.	640	426	COCO_train2014_000000407441.jpg
+566377	87850	a woman is dumping a bucket into a toilet	640	480	COCO_train2014_000000087850.jpg
+567125	45360	A zebra standing on top of a grass covered field.	459	640	COCO_train2014_000000045360.jpg
+569075	154435	Plate containing carrots, tomatoes, corn, potatoes and meat	424	640	COCO_train2014_000000154435.jpg
+570906	499147	The room is clean and ready for guests to use.	640	427	COCO_train2014_000000499147.jpg
+570912	314943	A train traveling down tracks next to a loading area.	427	640	COCO_train2014_000000314943.jpg
+573322	278790	A large jetliner sitting on top of an airport tarmac.	640	427	COCO_train2014_000000278790.jpg
+576145	97032	some kids are standing in a field playing sports	480	640	COCO_train2014_000000097032.jpg
+577724	452498	We are looking at a reflection on a TV.	471	640	COCO_train2014_000000452498.jpg
+580947	537135	a coach in front of a window with pillows	334	500	COCO_train2014_000000537135.jpg
+582933	431060	A black cat sitting on the back of a toilet tank.	640	427	COCO_train2014_000000431060.jpg
+585999	203486	A group of teddy bears lay in a basket.	333	500	COCO_train2014_000000203486.jpg
+588092	317962	A bed area with floral sheets and a cellphone on it.	427	640	COCO_train2014_000000317962.jpg
+588794	160931	A group of men standing on top of a baseball field.	480	640	COCO_train2014_000000160931.jpg
+588820	231786	a white icebox that is in some kitchen	640	480	COCO_train2014_000000231786.jpg
+588842	338910	A man riding a snowboard down a snow covered slope.	394	640	COCO_train2014_000000338910.jpg
+588877	11613	A couple of people racing each other on skis on top of snow.	429	640	COCO_train2014_000000011613.jpg
+590861	191570	A white toilet sitting in a room near a doorway.	640	427	COCO_train2014_000000191570.jpg
+592283	158757	A man kneeling down on his knees in front of a dog.	428	640	COCO_train2014_000000158757.jpg
+592724	347055	a person near a desk with a keyboard monitor and mouse	334	500	COCO_train2014_000000347055.jpg
+594636	207142	A ripe apple and pomegranate sit together on a table.	480	640	COCO_train2014_000000207142.jpg
+595163	3461	Two men standing on a hill in snow skis.	505	640	COCO_train2014_000000003461.jpg
+597035	569989	Two trays filled with bowls of food and beverages	640	427	COCO_train2014_000000569989.jpg
+597051	261761	a black and gray pigeon some windows and a building	612	612	COCO_train2014_000000261761.jpg
+597833	420231	A man is taking a bite of food	377	500	COCO_train2014_000000420231.jpg
+597845	324442	A stop sign with a couple more behind it	480	640	COCO_train2014_000000324442.jpg
+598673	347787	A computer loaded to a social networking site on a desk.	375	500	COCO_train2014_000000347787.jpg
+598799	128667	a lone train is sitting on the tracks at the station.	423	640	COCO_train2014_000000128667.jpg
+599026	396366	There is a bear that is crossing the river	426	640	COCO_train2014_000000396366.jpg
+600490	165654	A person is snowboarding down a snowy slope.	467	640	COCO_train2014_000000165654.jpg
+600686	178811	A young man wearing a hat also a suit and tie.	640	353	COCO_train2014_000000178811.jpg
+602927	403577	An animal is covering up the keyboard with it's long tail.	480	640	COCO_train2014_000000403577.jpg
+603788	208147	A bathroom with a toilet sitting next to a white sink.	500	375	COCO_train2014_000000208147.jpg
+604562	149215	A drip hose draped around several plants in a garden.	333	500	COCO_train2014_000000149215.jpg
+606682	121869	several male cows coming up a hill to cross the street	326	500	COCO_train2014_000000121869.jpg
+607085	353098	A room with cherry wood furniture and  TV on the stand	375	500	COCO_train2014_000000353098.jpg
+607701	405111	THERE IS A SIGN ON THE CORNER NEXT TO A STOP SIGN	375	500	COCO_train2014_000000405111.jpg
+611212	475169	Several pigeons eating food off of the sidewalk.	500	333	COCO_train2014_000000475169.jpg
+613388	401743	Baseball players posing for a picture before a game.	428	640	COCO_train2014_000000401743.jpg
+613400	526445	A baseball player hitting a ball and breaking his bat in the process.	488	640	COCO_train2014_000000526445.jpg
+613766	340129	A man taking picture of another man standing next to a baseball player.	427	640	COCO_train2014_000000340129.jpg
+613936	492671	An uncommon picture of a single animal inside this picture.	426	640	COCO_train2014_000000492671.jpg
+614269	428925	A pretty young lady holding a smart phone in her left hand.	427	640	COCO_train2014_000000428925.jpg
+615842	339111	An individual is caught in the hush of the image.	478	640	COCO_train2014_000000339111.jpg
+619634	62477	Two yellow taxis are on a dark city street.	427	640	COCO_train2014_000000062477.jpg
+620652	525813	A couple of cows laying on the grass next to a man wearing bicycle gear.	480	640	COCO_train2014_000000525813.jpg
+623322	430481	A lady is on her cellular phone while a Japanese sign.	375	500	COCO_train2014_000000430481.jpg
+624027	572554	A man eating a doughnut with a woman in a room.	424	640	COCO_train2014_000000572554.jpg
+624223	203794	a man standing at an outdoor table working on some pizzas	500	375	COCO_train2014_000000203794.jpg
+624621	26292	A man is pulling the string on a colorful kite, attempting to get it to fly.	480	640	COCO_train2014_000000026292.jpg
+625106	565002	A bearded man poses with his breakfast meal at a cafe	640	425	COCO_train2014_000000565002.jpg
+625908	198042	A professional baseball pitcher pitching to home plate.	427	640	COCO_train2014_000000198042.jpg
+628498	335865	a metal pitcher filled with some tulips	640	573	COCO_train2014_000000335865.jpg
+628880	116356	A man in a kitchen smiles holding a large drink lid.	480	640	COCO_train2014_000000116356.jpg
+631688	249839	A guy in a pizza place gives a thumbs up.	480	640	COCO_train2014_000000249839.jpg
+632659	499082	A bird sitting on a house eave in a backyard.	480	640	COCO_train2014_000000499082.jpg
+635606	492114	A woman posing in front of a display of baseball players at home plate.	480	640	COCO_train2014_000000492114.jpg
+637587	390567	Two food trucks are occupying spots near a sidewalk	478	640	COCO_train2014_000000390567.jpg
+638626	387604	Toast smothered in eggs and beans is a hearty meal.	478	640	COCO_train2014_000000387604.jpg
+642074	75084	A picture of a cat playing in some ones luggage.	480	640	COCO_train2014_000000075084.jpg
+644960	550532	A set of parking meters are seen outside a shop.	429	640	COCO_train2014_000000550532.jpg
+646266	127100	A motorcyclist stops on the road to allow a pedestrian to cross.	192	500	COCO_train2014_000000127100.jpg
+651665	548240	a cat that is hanging out of a car window	640	480	COCO_train2014_000000548240.jpg
+651985	344720	A few cars are sitting at a top light on a busy street.	640	432	COCO_train2014_000000344720.jpg
+654081	349103	Two people sitting at a table looking at a camera	290	448	COCO_train2014_000000349103.jpg
+656276	112443	The tennis player will hit the ball soon.	640	427	COCO_train2014_000000112443.jpg
+657434	404565	A group of skiiers skiing down a mountain.	640	427	COCO_train2014_000000404565.jpg
+663501	431793	there is a sandwich next to a bowl and spoon	640	480	COCO_train2014_000000431793.jpg
+666318	336607	A man running after a kite flying in the sky	480	640	COCO_train2014_000000336607.jpg
+667323	139359	A woman with glasses is putting something near a baby's mouth.	427	640	COCO_train2014_000000139359.jpg
+667340	218302	A black and a white horse are grazing in a green pasture.	480	640	COCO_train2014_000000218302.jpg
+675874	192831	Four trains sit on tracks in a rail yard in a city.	480	640	COCO_train2014_000000192831.jpg
+677450	426469	a man that is surfing on a blue surf board	514	640	COCO_train2014_000000426469.jpg
+677498	393730	A plate has a small amount of food on it	640	639	COCO_train2014_000000393730.jpg
+677552	268909	A young man holding a baseball bat on a green field.	640	427	COCO_train2014_000000268909.jpg
+680106	50018	A look into a very narrow looking kitchen with a black fridge.	500	375	COCO_train2014_000000050018.jpg
+681885	529055	a person hidden under an umbrella on a brick street	427	640	COCO_train2014_000000529055.jpg
+683425	470880	A boy on a skateboard doing a flip	500	333	COCO_train2014_000000470880.jpg
+684418	480052	Tagged cow getting a drink of water while standing in pasture.	375	500	COCO_train2014_000000480052.jpg
+685766	12728	A plane is parked on a airport runway.	480	640	COCO_train2014_000000012728.jpg
+686878	418180	A person who is performing a jump on snow skis.	427	640	COCO_train2014_000000418180.jpg
+686937	96130	An office desk chair sitting in front of a laptop computer.	375	500	COCO_train2014_000000096130.jpg
+687839	378710	A wall with four mounted urinals on it.	480	640	COCO_train2014_000000378710.jpg
+688866	231675	A little girl holding a brown teddy bear.	426	640	COCO_train2014_000000231675.jpg
+691418	440217	A large body of water with a train traveling over it.	361	640	COCO_train2014_000000440217.jpg
+694109	150185	A white refrigerator freezer sitting inside of a kitchen.	640	427	COCO_train2014_000000150185.jpg
+695364	67917	A couple of benches parked in top of sandy ground.	427	640	COCO_train2014_000000067917.jpg
+696603	59581	this is a man driving in a car	478	640	COCO_train2014_000000059581.jpg
+697984	89230	A zebra grazing on a grass covered field.	427	640	COCO_train2014_000000089230.jpg
+702020	298132	A group of three white teddy bears on top of a couch.	378	567	COCO_train2014_000000298132.jpg
+703711	124190	A giraffe standing on top of a lush green field.	640	512	COCO_train2014_000000124190.jpg
+703866	330708	Flying bird silhouetted overhead against cloudy sky background.	321	500	COCO_train2014_000000330708.jpg
+705578	296211	A skate boarder is doing some stunts on some steps.	427	640	COCO_train2014_000000296211.jpg
+708012	411083	A jar of honey with a plate of food in the background.	427	640	COCO_train2014_000000411083.jpg
+708905	218939	Two tennis players playing a game of tennis.	458	640	COCO_train2014_000000218939.jpg
+709307	313946	A pile of food sitting on top of a white plate.	480	640	COCO_train2014_000000313946.jpg
+709817	473131	A group of women sitting and standing around a bunch of ripe bananas.	480	640	COCO_train2014_000000473131.jpg
+711279	146030	Fluffy orange cat and big grey dog curled up together in a dog bed.	309	500	COCO_train2014_000000146030.jpg
+711544	544701	A man sitting in a chair looking at someone's food.	480	640	COCO_train2014_000000544701.jpg
+711602	527886	A man standing next to a bunch of snowboard.	400	500	COCO_train2014_000000527886.jpg
+712032	491743	A man standing outside of a building talking on a phone.	512	640	COCO_train2014_000000491743.jpg
+712446	329535	A clock on top of a pointed building.	640	427	COCO_train2014_000000329535.jpg
+715824	231744	a glass vase with some flowers growing out of it	640	427	COCO_train2014_000000231744.jpg
+717761	231677	A plate with a piece of food on top of it.	480	640	COCO_train2014_000000231677.jpg
+718510	171467	A pizza is displayed in an open box.	640	478	COCO_train2014_000000171467.jpg
+719880	250498	A plate topped with a sausage and rice.	480	640	COCO_train2014_000000250498.jpg
+719968	223119	A wooden fence sitting next to a lush green field.	612	612	COCO_train2014_000000223119.jpg
+720011	158099	A red and white train parked in a train station.	480	640	COCO_train2014_000000158099.jpg
+721026	340561	The Big Ben clock tower towering over the city of London.	372	500	COCO_train2014_000000340561.jpg
+721609	491976	A group of giraffe standing next to a fence.	425	640	COCO_train2014_000000491976.jpg
+721837	134262	A computer monitor sitting on top of a computer desk.	640	480	COCO_train2014_000000134262.jpg
+724617	210134	Several buses that are parked very close to each other in a parking lot.	480	640	COCO_train2014_000000210134.jpg
+731727	187200	A marine filled with lots of boats under a cloudy blue sky.	480	640	COCO_train2014_000000187200.jpg
+731931	472314	A cat lying half asleep on a chair with a teddy bear.	480	640	COCO_train2014_000000472314.jpg
+732747	166236	A group of men wading through a body of water.	458	640	COCO_train2014_000000166236.jpg
+733765	315053	A white plate topped with a cheesy sauce covered dish.	480	640	COCO_train2014_000000315053.jpg
+734384	153882	A blue stop sign  mounted to a wall.	375	500	COCO_train2014_000000153882.jpg
+734434	142016	A black bear laying on top of a field near trees.	427	640	COCO_train2014_000000142016.jpg
+736607	404537	The cell phone is next to a couple of leather pouches and a set of keys.	640	640	COCO_train2014_000000404537.jpg
+737518	192216	A white plate topped with meat, mac and cheese and a bun.	640	640	COCO_train2014_000000192216.jpg
+740521	176769	A teddy bear has a baby cup in front of it.	500	375	COCO_train2014_000000176769.jpg
+743464	326064	A well decorated bakers shelf with cups, and a vase of flowers.	640	640	COCO_train2014_000000326064.jpg
+747307	420617	A group of young people looking at computer images at an expo.	427	640	COCO_train2014_000000420617.jpg
+748541	417242	A train traveling past a building and two lights.	406	640	COCO_train2014_000000417242.jpg
+749382	319005	A view of buildings and cars on the street from under a rainbow colored umbrella.	480	640	COCO_train2014_000000319005.jpg
+752604	184550	A cellphone that is in front of a laptop.	375	500	COCO_train2014_000000184550.jpg
+752650	502620	The yellow construction vehicle is parked in the dirt.	640	427	COCO_train2014_000000502620.jpg
+754008	518180	A vase that has flowers inside of it.	429	640	COCO_train2014_000000518180.jpg
+758818	509931	A man touching an elephant with jewelry on trunk.	426	640	COCO_train2014_000000509931.jpg
+759435	122163	a person on a beach with a dog and a frisbee	428	640	COCO_train2014_000000122163.jpg
+762615	349932	Two men and two women on a sofa with laptop computers.	480	640	COCO_train2014_000000349932.jpg
+768541	253334	A person that is jumping in the air with skis.	426	640	COCO_train2014_000000253334.jpg
+769504	273658	There is a woven mat under the plate with the chocolate cheese cake on it.	480	640	COCO_train2014_000000273658.jpg
+769988	423858	A woman sitting on the bed holding a teddy bear.	426	640	COCO_train2014_000000423858.jpg
+772839	12664	A train passing down the track in the open.	416	640	COCO_train2014_000000012664.jpg
+775119	262013	The various wild animals are grazing in the pasture.	304	640	COCO_train2014_000000262013.jpg
+777027	113252	A man relaxes on his Kawasaki motorcycle on a sidewalk.	616	640	COCO_train2014_000000113252.jpg
+777181	11504	Sandwich served on plate at table with beverage.	375	500	COCO_train2014_000000011504.jpg
+778508	481257	A recreational vehicle the size of a bus parked in a parking lot.	438	640	COCO_train2014_000000481257.jpg
+790987	560978	Group of people posing for a picture holding cell phones.	431	640	COCO_train2014_000000560978.jpg
+792233	240618	The man is standing inside of the bathroom that is tore apart.	428	640	COCO_train2014_000000240618.jpg
+792266	454181	Students are sitting at tables with books and laptops.	640	480	COCO_train2014_000000454181.jpg
+792700	40449	A mirror view of a man taking a photo.	480	640	COCO_train2014_000000040449.jpg
+792859	360695	Two wooden bins filled with fruits and vegetables.	426	640	COCO_train2014_000000360695.jpg
+797664	465457	a big bowl of food with some bananas beside it	480	640	COCO_train2014_000000465457.jpg
+801622	54304	a close up of a person eating food at a table	612	612	COCO_train2014_000000054304.jpg
+803593	578306	the back of a passenger airplane with the wheels spreading out in both directions	427	640	COCO_train2014_000000578306.jpg
+803671	87847	A twisted sign in pile of rocks next to water.	640	426	COCO_train2014_000000087847.jpg
+804855	463102	Orange cupcake with Orange is sitting next to it.	640	427	COCO_train2014_000000463102.jpg
+805685	131172	A group of people are sitting on a bench outside.	469	640	COCO_train2014_000000131172.jpg
+808924	181909	a man is on a television with some bottles	480	640	COCO_train2014_000000181909.jpg
+809930	486595	a bunch of motor scooters are parked in a garage	480	640	COCO_train2014_000000486595.jpg
+811475	118086	A cat laying its arm across a persons arm.	640	480	COCO_train2014_000000118086.jpg
+812529	388611	A man rides a surf board in the water.	450	640	COCO_train2014_000000388611.jpg
+813097	525700	A man puts his arm around a woman sitting beside him on a bench.	424	640	COCO_train2014_000000525700.jpg
+813327	164178	The view of a living room couch and dresser with books on top.	480	640	COCO_train2014_000000164178.jpg
+829358	133071	Bread, sour cream, and guacamole are arrayed on a plate with a lemon slice.	360	640	COCO_train2014_000000133071.jpg

quant_sdxl/env/brv_sdxl_mlperf_minimal.yml

@@ -0,0 +1,19 @@
+name: brv_sdxl_mlperf
+channels:
+  - conda-forge
+dependencies:
+  - python=3.9
+  - setuptools<70
+  - pip:
+    - torch
+    - accelerate==0.23.0
+    - diffusers==0.21.2
+    - open-clip-torch==2.7.0
+    - opencv-python==4.8.1.78
+    - pycocotools==2.0.7
+    - scipy==1.9.1
+    - torchmetrics[image]==1.2.0
+    - tqdm
+    - transformers==4.33.2
+    - pandas==2.2.2
+    - brevitas[export] @ git+https://github.com/Xilinx/brevitas.git@fb2fc8703c7492566e0748138cfb046692e9b059

quant_sdxl/env/brv_sdxl_mlperf_rocm_minimal.yml

@@ -0,0 +1,21 @@
+name: brv_sdxl_mlperf_rocm
+channels:
+  - conda-forge
+dependencies:
+  - python=3.9
+  - setuptools<70
+  - pip:
+    - --index-url https://download.pytorch.org/whl/rocm6.0
+    - --extra-index-url https://pypi.org/simple
+    - torch
+    - accelerate==0.23.0
+    - diffusers==0.21.2
+    - open-clip-torch==2.7.0
+    - opencv-python==4.8.1.78
+    - pycocotools==2.0.7
+    - scipy==1.9.1
+    - torchmetrics[image]==1.2.0
+    - tqdm
+    - transformers==4.33.2
+    - pandas==2.2.2
+    - brevitas[export] @ git+https://github.com/Xilinx/brevitas.git@fb2fc8703c7492566e0748138cfb046692e9b059

requirements.txt → quant_sdxl/env/requirements.txt

@@ -7,4 +7,5 @@ scipy==1.9.1
 torchmetrics[image]==1.2.0
 tqdm
 transformers==4.33.2
-brevitas @ git+https://github.com/Xilinx/brevitas@dev
+pandas==2.2.2
+brevitas[export] @ git+https://github.com/Xilinx/brevitas.git@fb2fc8703c7492566e0748138cfb046692e9b059

minimal_script.py → quant_sdxl/quant_sdxl.py

@@ -32,11 +32,12 @@ from brevitas.graph.quantize import layerwise_quantize
 from brevitas.inject.enum import StatsOp
 from brevitas.nn.equalized_layer import EqualizedModule
 from brevitas.utils.torch_utils import KwargsForwardHook
+import brevitas.config as config
 
 from brevitas_examples.common.parse_utils import add_bool_arg
 from brevitas_examples.stable_diffusion.sd_quant.export import export_quant_params
 from brevitas_examples.stable_diffusion.sd_quant.nn import QuantAttention
-import brevitas.config as config
+from brevitas_examples.stable_diffusion.mlperf_evaluation.accuracy import compute_mlperf_fid
 
 TEST_SEED = 123456
 torch.manual_seed(TEST_SEED)
@@ -84,6 +85,8 @@ def main(args):
     dtype = getattr(torch, args.dtype)
 
     calibration_prompts = load_calib_prompts(args.calibration_prompt_path)
+    assert args.calibration_prompts <= len(calibration_prompts) , f"--calibration-prompts must be <= {len(calibration_prompts)}"
+    calibration_prompts = calibration_prompts[:args.calibration_prompts]
     latents = torch.load(args.path_to_latents).to(torch.float16)
 
     # Create output dir. Move to tmp if None
@@ -91,6 +94,7 @@ def main(args):
     str_ts = ts.strftime("%Y%m%d_%H%M%S")
     output_dir = os.path.join(args.output_path, f'{str_ts}')
     os.mkdir(output_dir)
+    print(f"Saving results in {output_dir}")
 
     # Dump args to json
     with open(os.path.join(output_dir, 'args.json'), 'w') as fp:
@@ -122,6 +126,20 @@ def main(args):
             raise RuntimeError("LoRA layers should be fused in before calling into quantization.")
 
     pipe.set_progress_bar_config(disable=True)
+
+    if args.load_checkpoint is not None:
+        with load_quant_model_mode(pipe.unet):
+            pipe = pipe.to('cpu')
+            print(f"Loading checkpoint: {args.load_checkpoint}... ", end="")
+            pipe.unet.load_state_dict(torch.load(args.load_checkpoint, map_location='cpu'))
+            print(f"Checkpoint loaded!")
+        pipe = pipe.to(args.device)
+
+    if args.load_checkpoint is not None:
+        # Don't run full activation equalization if we're loading a quantized checkpoint
+        num_ae_prompts = 2
+    else:
+        num_ae_prompts = len(calibration_prompts)
     with activation_equalization_mode(
             pipe.unet,
             alpha=args.act_eq_alpha,
@@ -135,7 +153,7 @@ def main(args):
         total_steps = args.calibration_steps
         run_val_inference(
             pipe,
-            calibration_prompts,
+            calibration_prompts[:num_ae_prompts],
             total_steps=total_steps,
             test_latents=latents,
             guidance_scale=args.guidance_scale)
@@ -174,7 +192,7 @@ def main(args):
     nn.Linear: (qnn.QuantLinear, quant_linear_kwargs),
     nn.Conv2d: (qnn.QuantConv2d, quant_layer_kwargs),
     'diffusers.models.lora.LoRACompatibleLinear':
-        (LoRACompatibleQuantLinear, quant_layer_kwargs),
+        (LoRACompatibleQuantLinear, quant_linear_kwargs),
     'diffusers.models.lora.LoRACompatibleConv': (LoRACompatibleQuantConv2d, quant_layer_kwargs)}
 
     pipe.unet = layerwise_quantize(
@@ -183,28 +201,35 @@ def main(args):
 
     pipe.set_progress_bar_config(disable=True)
 
-    print("Applying activation calibration")
-    with torch.no_grad(), calibration_mode(pipe.unet):
-        run_val_inference(
-            pipe,
-            calibration_prompts,
-            total_steps=args.calibration_steps,
-            test_latents=latents,
-            guidance_scale=args.guidance_scale)
-
-    print("Applying bias correction")
-    with torch.no_grad(), bias_correction_mode(pipe.unet):
-        run_val_inference(
-            pipe,
-            calibration_prompts,
-            total_steps=args.calibration_steps,
-            test_latents=latents,
-            guidance_scale=args.guidance_scale)
-
-    if args.checkpoint_name is not None:
-        torch.save(pipe.unet.state_dict(), os.path.join(output_dir, args.checkpoint_name))
+    if args.load_checkpoint is None:
+        print("Applying activation calibration")
+        with torch.no_grad(), calibration_mode(pipe.unet):
+            run_val_inference(
+                pipe,
+                calibration_prompts,
+                total_steps=args.calibration_steps,
+                test_latents=latents,
+                guidance_scale=args.guidance_scale)
+
+        print("Applying bias correction")
+        with torch.no_grad(), bias_correction_mode(pipe.unet):
+            run_val_inference(
+                pipe,
+                calibration_prompts,
+                total_steps=args.calibration_steps,
+                test_latents=latents,
+                guidance_scale=args.guidance_scale)
+
+        if args.checkpoint_name is not None:
+            torch.save(pipe.unet.state_dict(), os.path.join(output_dir, args.checkpoint_name))
+
+    # Perform inference
+    if args.validation_prompts > 0:
+        print(f"Computing validation accuracy")
+        compute_mlperf_fid(args.model, args.path_to_coco, pipe, args.validation_prompts, output_dir)
 
     if args.export_target:
+        pipe.unet.to('cpu').to(dtype)
         export_quant_params(pipe, output_dir)
 
 
@@ -220,6 +245,23 @@ if __name__ == "__main__":
         '-d', '--device', type=str, default='cuda:0', help='Target device for quantized model.')
     parser.add_argument(
         '--calibration-prompt-path', type=str, default=None, help='Path to calibration prompt')
+    parser.add_argument(
+        '--calibration-prompts',
+        type=int,
+        default=500,
+        help='Number of prompts to use for calibration. Default: %(default)s')
+    parser.add_argument(
+        '--validation-prompts',
+        type=int,
+        default=0,
+        help='Number of prompt to use for validation. Default: %(default)s')
+    parser.add_argument(
+        '--path-to-coco',
+        type=str,
+        default=None,
+        help=
+        'Path to MLPerf compliant Coco dataset. Required when the --validation-prompts > 0 flag is set. Default: None'
+    )
     parser.add_argument(
         '--checkpoint-name',
         type=str,
@@ -228,12 +270,16 @@ if __name__ == "__main__":
         'Name to use to store the checkpoint in the output dir. If not provided, no checkpoint is saved.'
     )
     parser.add_argument(
-        '--path-to-latents',
+        '--load-checkpoint',
         type=str,
         default=None,
+        help='Path to checkpoint to load. If provided, PTQ techniques are skipped.')
+    parser.add_argument(
+        '--path-to-latents',
+        type=str,
+        required=True,
         help=
-        'Load pre-defined latents. If not provided, they are generated based on an internal seed.')
-
+        'Path to pre-defined latents.')
     parser.add_argument('--guidance-scale', type=float, default=8., help='Guidance scale.')
     parser.add_argument(
         '--calibration-steps', type=float, default=8, help='Steps used during calibration')