This is a submission for the The Pinata Challenge

What I Built

There are many great artists out there, that don't get to display their art works in an art gallery or museum as they can be expensive for artists as well as because of the selection criteria.

So, I built a virtual art museum/gallery using Three.js, Pinata and expressjs to help artists display their artwork in an art gallery. Also allowing people to explore other people artwork in a virtual museum/gallery.

Demo

Try on non-touch devices for best experience

• Use WASD to move, Space to jump (not necessary though)
• Mouse to drag to look around.
• To hide all annotations, press shift key.
• To upload, click on the annotation (number).

• To see details of the art-work hover your mouse over the annotation.

• Use Escape key to bring up menu and switch between galleries.

• If annotations or the images are not loading, you have hit the rate limit. Wait for few minutes and try again or clone the repo and try locally.

Note: you'll have to wait sometime to allow the models to load,
will compress them soon :)

Link: Virtual gallery

Major features

• Upload artwork and view artworks within 3d models.
• Cropper aspect depending on the frame aspect ratio.
• Allow crediting art work via twitter handle.
• Annotation to get quick info of the artwork.
• First person view

Screen shots

Without images, upload button

with Images

Upload image

Annotation explaining details

Other model

Menu

My Code

I put the code together in a bit of a hurry, so refactoring is needed. However, the code available is open-sourced, both backend and frontend in the same directory.

I am also working on a museum model, if there's time left, I'll add them too. Until then feel free to check out the existing model and code

git clone https://github.com/PaulleDemon/art-museum

SUPABASE_URL="https://.supabase.co"
SUPABASE_KEY=""

PINATA_API_KEY=""
PINATA_SECRET_API_KEY=""

PINATA_JWT=""

cd backend
node index.js

BACKEND_URL="http://localhost:3000"

PROD_BACKEND_URL="https://"

npm run start

More TechnicalDetails

• Pinata was used to upload the arts to IPFS, the IPFS hash is then stored in the database.

• The annotation are displayed using Css2dobject and CSS2drenderer from Three.js

• Model credits goes to great artists from SketchFab, licensed under CC-4.0. Each model contains the license inside them

• I did the UV unwrapping and separated the art mesh using Blender.

I created the database in supabase and added the following fields

id -> auto
img_id -> string // this is the id of the image holder mesh, generally mesh.name
museum -> 0 | 1 | 2 // 0 is image gallery 2 is museum model.
title -> string // title of the image
description -> string // description of the image
name -> string //artist name, curently twitter handle
img_cid -> string // IPFS id returned after uploading to pinata
price -> number // if the artist wishes to sell the art, asking price
created_at -> datetime // datetime when the row was inserted

Challenges

• Among the hard parts were trying to load the GLTF files and resolve the paths using webpack. Eventually, I decided to just put the gltf files inside the public folder.

• Working on first person viewer also was quite challenging, since you need to account for collisions, but it became a bit easier once I studied some examples in the threejs docs.

• Unlike adding css image contain, it's super challenging to get the images to contain inside the mesh, since the texture would vary depending on the UV unwrapping and it can become hard to get the image to contain inside the mesh.

Tech stack / Libraries

• Three.js - 3d models, loading models, first person view etc.
• Express.js - backend api
• Supabase - database
• Pinata - file upload

Future vision of this Virtual gallery.

• Enable VR mode.
• Add couple more art galleries and museums models.
• Allow artists to sell their artworks via the gallery.