AI-Based Representation: Diffusion Models Fine-tuning as a Way of Transformative Operative Èkphrasis

This study is positioned at the intersection of architectural representation and GenAI (generative artificial intelligence), critically and systematically exploring the transformative potential of fine-tuning DMs (diffusion models). Starting from acknowledging the intrinsic limitations of pure prompt engineering, this research posits that fine-tuning emerges as a contemporary form of transformative operative èkphrasis. This approach enables personalised conditioning in the generation of architectural imagery. This contribution critically evaluates their respective potentials and limitations through a comparative analysis of prominent methodologies such as DreamBooth, Textual Inversion, LoRA, and Hypernetworks. It also highlights the inherent challenges stemming from the two-dimensional nature of the generated visual representations. Emphasising the necessity for developing evaluation metrics calibrated explicitly for the architectural domain, this research outlines pertinent future research directions. These encompass seamless integration with BIM/CAD environments, the exploration of 3D model generation, and empirical validation through operational experimentation. Fine-tuning is positioned as an enabling technological tool for innovation within architectural representation.