Writer : Nikolaos Partarakis, PhD, Effie Karuzaki (Mrs.), Paraskevi Doulgeraki (Mrs.), Carlo Meghini (Mr.), Cynthia Beisswenger (Mrs.), Hansgeorg Hauser (Mr.), Xenophon Zabulis, PhD
Year : 2021
According to the European Policy Brief of the RICHES project, however, some isolated positive signs may give rise to an increase in handmade products in the future. For example, do-it-yourself culture, the emergence of digital fabrication at home (e.g. 3D printing) and hybrid forms of making, which merge the traditional with the contemporary, are giving life to several small businesses. At the same time, the rising need for individuality in consumer societies has increased the demand for unique, customised products. In this context, a repositioning of craft skills, techniques, patterns and materials is possible by augmenting them through the integration of digital technology. In this article, we argue that this may lead to a new form of craft-based innovation, which can also be promoted through ethical approaches to sustainability, ecologic use and local production.
According to Kurin (2007), ‘Intangible Cultural Heritage (ICH) – a loose English translation of the Japanese mukei bunkazi, is broadly defined in terms of oral traditions, expressive culture, the social practices, ephemeral aesthetic manifestations, and forms of knowledge carried and transmitted within cultural communities’. Heritage crafts, a form of ICH, are strongly bound to the history and traditions of the communities in which they are practised (Kurin 2007), but they are also linked to the history and tradition of humanity as a whole, through common links and craft manifestations across the globe.
In this work, we focus on the craft of textile weaving and the presentation of the historic patterns, symbols and motifs through storytelling. Like many craft products, in traditional textiles, one can often recognise the origin of textile from its decorative patterns or its visual style. The pattern of a woven product may contain an aesthetic, decorative or artistic dimension. Weaving a patterned fabric is a tedious process that requires additional skill to encode specific multicoloured patterns. The invention of mechanical looms increased the automation of the tedious task of patterned weaving and resulted in the ability to manufacture highly detailed patterns more quickly and with fewer errors.
The relation of weaving to storytelling and music is surprisingly old. For example, Euripides’s consistent pairing of weaving with the recitation of mythological stories is intriguing:
As the texts suggest, women retell stories while weaving, and this means of retelling is differentiated from other modes of storytelling. If the processes of relating such myths were directly associated with patterning in the textiles as they are produced, then the recitation would necessarily embed numerical information about the thread counts of the woven pattern. (Tuck 2009)It remains a mystery today whether storytelling provided thematology or acted as a numeric code corresponding to groups of knots of a given colour or the number of warp threads overshot by those of the weft. Regardless of the kind of connection, it is still important today to revive memories and stories carried by traditional motifs, patterns and symbols used in TCs (Heckman and Heckman 2003).
To support this vision, this work presents the design, implementation and augmentation of a handmade handbag, to act as a storyteller for the historic patterns of Haus der Seidenkultur (HdS), in the city of Krefeld, Germany. From a technological point of view, this research work exploits the evolution of augmented reality (AR), virtual reality (VR) and mixed reality (MR), which tries to identify and design new forms of interaction with cultural heritage artefacts, thus providing an alternative approach for experiencing handmade products through storytelling and mobile devices.
The history of Krefeld is closely linked to this magical material. Today, there is a small museum, the HdS, which shows how the history of silk has shaped the development of the town over the past three centuries. Krefeld is a town with a Roman past. Excavations in neighbouring Gellup revealed the existence of at least six Roman camps up to the fourth century AD. Burial sites dating to the time of the Franks indicate that the area was settled continuously. Starting in the late Middle Ages, Krefeld belonged to the County of Moers. The Counts of Moers made every effort to establish the Reformation early in their territory; the first Reformist preacher in the area took up his post in 1561. At the beginning of the 17th century, Krefeld came under the rule of the Netherlands, and the town became an island of religious tolerance. Consequently, in a period in which the denomination of the population was determined by the denomination of the ruler, Mennonites from near and far came to Krefeld and settled there. This immigration had far-reaching consequences that have shaped the profile of the town to the present day.
The religious refugees brought with them linenprocessing skills, and, because they were also successful business people, they laid the foundations for economic growth and prosperity. The Von der Leyen family,
immigrants from Radevormwald, also contributed significantly to the development of the ‘Town like Silk and Velvet’. Originally linen weavers, the family increasingly changed the emphasis of their business to silk weaving. In 1702, Krefeld came under Prussian rule, and silk weaving became the most important economic factor: sales to the Prussian court in Berlin flourished. During this period, the silk weavers were out workers who received orders from merchants and traders to weave fabrics. The looms were set up in front of the light window in the typical small cottages, some of which still exist today. The head of the household was normally the weaver, and other family members helped with tasks such as reeling the thread onto the bobbins for the shuttle. On one of the main avenues of the town, there is a monument to the weaver ‘Meister Ponzelaar’. He wears his Sunday best: a frock coat (in the local dialect, a ‘Laakesserock’), a high-necked waistcoat, a small collar with a silk scarf and a ‘Jraduutkapp’ (a black cap). At the end of the week, he takes the finished fabric on the beam, together with a bag containing any leftover thread, to the merchant’s office. There, he was paid and received a new prepared warp beam and thread for the week ahead. Such weavers were a typical sight in the town until the beginning of the 19th century. Their craft required rapid comprehension and rhythmic movement of hand and foot.
In 1785, Edward Cartwright invented his first mechanical loom and continued to make improvements to it. The enhanced looms became available on the market in 1820. With the advent of mechanisation, the silk entrepreneurs started to build factories where all the machines were powered by one source of energy and the workers were responsible for more than one loom.
Our approach is rooted in the idea that humans exhibit a limited capacity to memorise inventories, such as a list of events as opposed to stories imbued with meaning or causal dependencies (Straub 2008). In this work, an approach is proposed that binds contemporary craft products with the cultural context of their depicted patterns, symbols and motifs through stories that explain their elements. As such, among the available technologies, augmented reality (AR), virtual reality (VR) and mixed reality (MR) can empower experiences in multiple contexts of use and can be gracefully used to augment physical items or their digital representation with information.
However, one of the earlier manifestations of AR regards its exploitation in the context of cultural heritage objects and sites, aiming to improve the visitors’ experience in these spaces. For example, mobile museum guides use AR technology to enrich exhibits with information (Wojciechowski et al. 2004; Schavemaker et al. 2011; Schmalstieg and Wagner 2005; Sidyawati et al. 2013). This approach has also been applied at archaeological sites, giving rise to new forms of digitally augmented tours (Stricker et al. 2001; Vlahakis et al. 2001; Kourouthanassis et al. 2015). Virtual restoration of religious heritage objects through 3D AR technology content has also been explored (Gîrbacia et al. 2013; Narciso et al. 2015). In the same vein, mobile AR has been used to enhance archaeological sites with more than static information. For example, virtual and real scenes at the site of ancient Pompeii have been mixed, so virtual avatars can present moments from everyday life in Pompeii (Papagiannakis et al. 2002; Papagiannakis et al. 2005; Papagiannakis et al. 2006). One of the major opportunities for the penetration of mobile AR technologies in the museum context is the low cost of such solutions because no equipment is required from the museum side (Angelopoulou 2011). According to the same research, museums have started to realise that AR can be an effective way to build user interest in museum collections and exhibits. AR is considered to be a way to preserve history, enhance visitor satisfaction, generate positive word-of-mouth, attract new target markets and contribute to a positive learning experience. AR also has an economic, experiential, social, epistemic, cultural, historical and educational value from stakeholders’ perspectives (Tom Dieck and Jung 2017).
3D models and 3D reconstructions are useful to preserve information about historical artefacts. However, the potential of this digital content is not fully realised, because it is not used to communicate information to museum visitors (Gonizzi Barsanti et al. 2015). A recent example is the exploitation of 3D reconstruction in the context of 3D games that facilitate exploration and learning (Partarakis et al. 2020). In this context, VR can be exploited to bridge the gap between the virtual representation of an artefact and the visitor, allowing multiple forms of interaction and storytelling with cultural heritage in the virtual or physical museum setting (Bruno et al. 2010; Carrozzino and Bergamasco 2010).
Taking into account the above issues, a careful study of visual aesthetics was needed to decide upon the design approach. Initially, it was decided on the shape of the bag. Taking into account that the handbag should contain several patterns, the size of the bag was decided to be approximately 35 by 40 centimetres. The size was selected following the size of the weaved patterns. Then, based on the size, several sketches of the bag were made to further discuss the concept and help determine the spatial arrangement of patterns. Some preliminary sketches from various angles are presented in Figure 2. As shown in these sketches, it was decided to make the bag minimal in terms of design and ensure that enough negative space was left on both sides to host patterns. Furthermore, this minimal approach would minimise the possibility of visual clutter between the artefact and the artistic qualities of the textile fabrics. The objective was to make the bag ‘invisible’ in terms of aesthetics to transmit the aesthetic qualities of the subject matter depicted by the patterns. Additionally, it was decided that the bag should have a big, solid base, to ensure that it would stand when placed on a surface, and a long strap, so the bag could be worn on a shoulder but still be visible.
Then, based on the preliminary sketches of the bag, it was decided that the patterns should be constrained to a strict geometric approach, so rectangular pieces of fabric would be selected to maintain a common visual feature among the patterns. Furthermore, it was decided that some form of neutral separator between the patterns was required to provide some form of visual arrangement. A black stripe, for example, could provide a good separator between colourful patterns. Third, it was decided that negative space (visually empty space) would further differentiate areas of interest from ones not visually interesting.
Based on the above decisions, it was decided that a concept rooted in geometric abstraction, as manifested in 20th-century art history, could provide a structural abstraction and thus a base for the aesthetic design of the object. This resulted in the selection of Piet Mondrian’s abstract style. In his paintings, primary colours are orchestrated in strong geometrical abstraction, providing time and space for the eye to experience the patterns and colours.
To do so, first, the market was identified as one of the contemporary handcrafted products that have both an artistic and historical dimension. People that select such products are not searching just for aesthetics but also stories. In this context, the story of this bag can be marketed under the following catchphrase: ‘A handbag to carry and to experience’.
The outcome of this process is summarised as follows:
the handbag is a practical accessory to carry things. But when the bag is not being carried, can it be experienced? The answer is that, in principle, yes, but through which technology? What is the most common digital accessory placed in a handbag? And the answer was apparent: the smartphone!
Following this rationale, the digital concept was conceptualised. We started from the principle that the handbag contains stories waiting to be discovered with your mobile phone. The mobile phone gives life to the handbag via an AR application. The idea was to create a piece of craftsmanship that is not only aesthetically pleasing but also a storyteller.
The interior of the handbag should provide stiff support to the exterior to be a bag that can stand on its own when placed on a surface and provide a smooth surface on the exterior to eliminate possible deformations of the patterns. It is important that the patterns remain visible and with limited deformation to support the recognition algorithms. Another requirement for the interior was that it be waterproof. This is a practical provision to support the actual usage of the artefact as a handbag. For the interior, a stiff, waterproof, synthetic material was used. The same combination of canvas and synthetic material was used for the handbag straps to give a consistent appearance and feel as the rest of the handbag.
After the selection of material, the actual craft creation was initiated. Initially, the pieces of the handbag were designed in semi-transparent paper and then transferred to the fabric layers. Then, the abstraction of the selected Mondrian composition was sketched on both sides of the exterior fabric in water-soluble pencil. The black stripes were then painted by hand, on top of the sketch, with a black textile marker to create the outline of the composition.
In this phase, the creation process was paused to proceed with the selection of patterns in conjunction with the abstract design of the handbag. Although the design does not exhibit the vividness of the pure primary colours used by Mondrian, the selection was made to create maximum visual contrast and thus remain in the spirit of the original creation. An example of pattern placement evaluation is presented in Figure 3. The rationale of the selection of pattern placement is twofold. First, we were interested in visual stimuli; thus, placement should produce stimulus input to the eye of the viewer, and this is done through contrast variations. In the case of patterns with limited contrast to each other, negative space was integrated to support the visual differentiation of patterns. Second, we were interested in the visual contrast of the greyscale representation of patterns. Such visual contrast is needed to assist the image-processing algorithms in differentiating patterns that are not separated through negative space and thus could coexist in the visual field of an application. Finally, to ensure the patterns were not curled on the sides of the bag, spacers had been integrated on the sides of the composition to ensure that visible patterns remained on the main planes of the bag.
After the selection of patterns, their cropping was decided based on their placement to ensure that the patterns would remain visible after cropping. Then, the patterns were sewed on both sides of the bag and the bag straps, and the bag was assembled to produce the final artefact as shown in Figure 4.
According to ARCore specifications, every image in this database should meet the following guidelines:
1\. Images should avoid repeating motifs and large numbers of geometric features – or very few features\. 2\. The images should have a relatively high contrast when turning into greyscale because no colour information is used by the identification algorithms of ARCore. 3\. The images should be flat \(for example\, not wrinkled or wrapped around a bottle\)\. 4\. All images should feature a resolution of at least 300 x 300 pixels\. Using images with high resolution does not improve performance\.
Initially, however, the application was not able to recognise most of the motifs. This was because ARCore uses a fixed focus, so the camera image in the ARCore application was blurry unless the phone was held at a specific distance from the bag. This problem was partially solved by adding a <ARCoreSession> module to the camera that enables setting the camera focus mode to ‘auto’. Using autofocus, the camera could focus on the target motifs on the bag better, and the application was able to identify the patterns a little better than before – but it still was not good enough. After a lot of trials, it was clear the reason for the low recognition rates was the quality of the images provided to the database. Indeed, guideline four explicitly states that ‘using images with high resolution does not improve performance’. Using images with high resolution hindered the performance of the ARCore recognition algorithms. To solve this problem, software was used to lower the images’ quality, although ensuring that the shortest side of the image would be 300 pixels. For example, one image was transformed from 3024 x 4032 pixels to 300 x 400 pixels. In this way, the ARCore performed significantly better in successfully recognising images; however, the application needed more than a second to do so. When an image is initially detected by ARCore, and no expected physical size has been specified, the ARCore will try to specify its size automatically. However, when providing the library with the actual physical dimensions of the target images, the recognition of the image is much faster. In this vein, the physical width of the motifs was measured, and these values were fed to ARCore. Indeed, this action boosted the ARCore performance, and the application successfully recognised the patterns relatively quickly.
Sequences of events and their presentation provide the means for the definition of (a) fabulae that are considered the sequence of historic events contributing to a story, (b) narratives that are stories based on a fabula, (c) narrations that are different forms of telling the same story and (d) presentations that define how the story will be told according to the target audience and technology used. Historical narratives can be exported and presented in a textual form by the AR app or previewed as multimodal documents through MOP. An example of the latter case is presented in Figure 7 and regards the story of textile weaving in Krefeld.
The AR app augments the virtual space with a canvas to present those stories. Furthermore, as these stories relate to the history of Europe and the social and historic dimension of textile weaving, the bag is also a portal to a web of information provided through the MOP. To reach this information, a button with the words ‘view more info’ navigates the user to the respective webpage of the MOP – where the journey in European history begins.
Overall, the authors argue that this process has proven their hypothesis that digital technology can be gracefully combined with traditional handmade artefacts to provide the possibility for a new form of entrepreneurship concerning the history and tradition of TCs and, at the same time, acknowledgement of the role of digital technologies in modern societies. Currently, the final artefact is intended to act as a physical exhibit at the museum, where it will be experienced by visitors as a storyteller. Furthermore, elaboration of the AR app will be done through the user-based evaluation, which will accompany the pilot instantiation of Mingei technology in the museum.