关于深度学习:使用Stable-Diffusion和Pokedex的描述生成神奇宝贝图片

53次阅读

共计 4965 个字符，预计需要花费 13 分钟才能阅读完成。

还记得咱们以前应用 GAN、Clip、DALL- E 生成神奇宝贝的文章吗，当初是时候应用 Stable Diffusion 了

在本文中，我将展现如何从神奇宝贝系列不同游戏中的 Pokedex 条目中获取神奇宝贝形容，并应用 Stable Diffusion 依据这些蔑视生成图片，这样能够看看 AI 如何解释这些形容的。这篇文章中，我只生成了最后的 150 个神奇宝贝，如果须要其余的能够自行尝试。

第一件事是取得 Pokedex 的形容。这些 Pokedex 的形容将作为生成图片的文本提醒。我最后的想法是为 pokemon.com 编写一个 webscraper，Pokedex Number (NPN) 执行搜寻查问。这并不是很难做到，然而有一个叫做 PyPokedex 的小 Python 库十分的好用，如果你对编程和神奇宝贝感兴趣，我倡议你去看看!

PyPokedex 应用 PokeAPI 来获取神奇宝贝数据，所以能够应用 PyPokedex 取得各种信息，如 NPN，名称，身高，体重，类型，根底统计等。除此之外 PyPokedex 还有一个叫做“get_description”的办法，它以字典的模式返回每个不同游戏的 Pokedex 形容。

一个根本的 PyPokedex 查问如下所示:

 pokemon=pypokedex.get(dex=poke_id)

这个查问会返回一个实例化的对象，该对象将蕴含基于 NPN 值的给定神奇宝贝的所有信息。

 poke_name=pokemon.name

下一步就是用 get_descriptions 办法获取 Pokedex 形容。这里咱们应用《Pokemon Yellow》的 Pokedex 中的形容，因为这代大家都应该晓得

 yellow_description=pokemon.get_descriptions()[ver]

而后将这两个后果联合起来，这样给 AI 的提醒就好了:

 prompt = poke_name + " " + yellow_description

这里咱们应用神奇宝贝的名字作为提醒的一部分，因为有一些 Pokedex 条目有点含糊，并不总是产生好后果。所以神奇宝贝的名字，可能发明出更像是原版的“真正变体”。如果你不喜爱也能够去掉，但我对这样的后果很称心。

咱们曾经有文本了，上面开始筹备 Stable Diffusion 模型。用上面这行代码装置必要的包:

 %pipinstall—quiet—upgradediffuserstransformersacceleratemediapyscipyftfyspacy

还须要蕴含 xformers 包来帮忙咱们创立图像。这个须要依据你的 python 版本训责相应的 wheels 包，肯定不要错了

 github_url="https://github.com/brian6091/xformers-wheels"
 xformer_id="0.0.15.dev0+4c06c79"
 xformers_wheels=f"xformers-{xformer_id}.d20221205-cp38-cp38-linux_x86_64.whl
 %pipinstall-q {github_url}/releases/download/{xformer_id}/{xformers_wheels}

接下来，指定想要应用的模型。这里抉择 dreamlike-photoreal-2.0。

 model_id=“dreamlike-art/dreamlike-photoreal-2.0”

咱们这里应用简略，疾速的办法：间接应用 StableDiffusionPipeline 生成图片

 importmediapyasmedia
 importtorch
 fromdiffusersimportStableDiffusionPipeline
 
 device="cuda"
 
 ifmodel_id.startswith("stabilityai/"):
   model_revision="fp16"
 else:
   model_revision=None
 
 ifschedulerisNone:
   pipe=StableDiffusionPipeline.from_pretrained(
       model_id,
       torch_dtype=torch.float16,
       revision=model_revision,
       )  
 else:
   pipe=StableDiffusionPipeline.from_pretrained(
       model_id,
       scheduler=scheduler,
       torch_dtype=torch.float16,
       revision=model_revision,
       )
 
 pipe=pipe.to(device)
 pipe.enable_xformers_memory_efficient_attention()
 
 ifmodel_id.endswith('-base'):
   image_length=512
 else:
   image_length=768

当初曾经筹备好所有步骤了，上面开始正式生成了：

 remove_safety=False
 num_images=4
 
 ifremove_safety:
   negative_prompt=None
 else:
   negative_prompt="nude, naked"
 
 images=pipe(
     prompt,
     height=image_length,
     width=image_length,
     num_inference_steps=25,
     guidance_scale=9,
     num_images_per_prompt=num_images,
     negative_prompt=negative_prompt,
     ).images
     
 media.show_images(images)

下面的代码曾经能够依据单个查问生成单个图片。然而咱们须要解决 150 个。所以我将他们整合成一个函数，循环调用。

 defmakePokemonFromPokedex(ver,nPokemon):
   #Loop over nPokemons to get the descritptions and generate images for each
   #poke_id = 1
   forpoke_idinrange(1, nPokemon+1, 1):
     #print(poke_id)
     #Specify which Pokemon we want to query using its ID number
     pokemon=pypokedex.get(dex=poke_id)
     #print(pokemon)
 
     #This is the name of the Pokemon we are querying
     poke_name=pokemon.name
 
     #This is the PokeDex desciption for the current Pokemon
     yellow_description=pokemon.get_descriptions()[ver]
 
     #This is the prompt I'll feed to the AI 
     prompt=poke_name+" "+yellow_description
     #print(prompt)
 
     remove_safety=False
     num_images=4
 
     ifremove_safety:
       negative_prompt=None
     else:
       negative_prompt="nude, naked"
 
     images=pipe(
         prompt,
         height=image_length,
         width=image_length,
         num_inference_steps=25,
         guidance_scale=9,
         num_images_per_prompt=num_images,
         negative_prompt=negative_prompt,
         ).images
     
     fname='poke_'+str(poke_id)
     get_concat_h_multi_blank(images).save(fname+'.jpg')

上面看看后果

铁甲蛹

Metapod — The prompt for this is“It is waiting for the moment to evolve. At this stage, it can only harden, so it remains motionless to avoid attack”.

独角虫

Weedle —T he prompt for this is“Beware of the sharp stinger on its head. It hides in grass and bushes where it eats leaves”.

大针蜂

Beedrill— The prompt for this is“It has three poisonous stingers on its forelegs and its tail. They are used to jab its enemy repeatedly”.

阿柏蛇

Ekans— The prompt for this is“The older it gets, the longer it grows. At night, it wraps its long body around tree branches to rest.”.

穿山鼠

Sandshrew— The prompt for this is“It loves to bathe in the grit of dry, sandy areas. By sand bathing, the Pokémon rids itself of dirt and moisture clinging to its body.”. Image by author.

穿山王

Sandslash— The prompt for this is“The drier the area Sandslash lives in, the harder and smoother the Pokémon’s spikes will feel when touched.”. Image by author.

尼多力诺

Nidorino— The prompt for this is“With a horn that’s harder than diamond, this Pokémon goes around shattering boulders as it searches for a moon stone.”. Image by author.

九尾

Ninetales— The prompt for this is“It is said to live 1,000 years, and each of its tails is loaded with supernatural powers”. Image by author.

走路草

Oddish— The prompt for this is“If exposed to moonlight, it starts to move. It roams far and wide at night to scatter its seeds.”. Image by author.

地鼠

Diglett— The prompt for this is“If a Diglett digs through a field, it leaves the soil perfectly tilled and ideal for planting crops.”. Image by author.

蚊香蝌蚪

Poliwag— The prompt for this is“For Poliwag, swimming is easier than walking. The swirl pattern on its belly is actually part of the Pokémon’s innards showing through the skin.”. Image by author.

毒刺水母

Tentacruel— The prompt for this is“When the red orbs on Tentacruel’s head glow brightly, watch out. The Pokémon is about to fire off a burst of ultrasonic waves.”. Image by author.

https://avoid.overfit.cn/post/7f072f5e107145d0b6502a956d4e3ede

作者：Victor Murcia

正文完