MySQL : 3 prinsip dasar optimisasi

:: Ada klien saya yang minta agar server databasenya dipindahkan ke server lainnya yang ada SSD nya. Nampaknya ybs berharap bahwa SSD akan bisa menjadi solusi instan untuk lambatnya beberapa query yang ada.

Sayangnya, untuk berbagai query yang buruk, seringkali solusinya hanya satu : ganti dengan query yang bagus :)

Ketika ada banyak user yang melakukan banyak query, dan server menjadi lambat, SSD bisa menjadi solusi untuk mempercepatnya.
Karena kapasitas IOPS (Input/Output Per Second) yang jauh lebih tinggi daripada hard disk.

Namun jika hanya ada beberapa query dan lambat, maka SSD bukan solusinya. Query nya yang harus di optimize :)

Saya sudah pernah mengalami sebuah query yang buruk, yang bahkan berjalan sangat lambat di RAM :D kalau di SSD sudah pasti akan lebih lambat lagi, he he…. saya tuliskan sekilas disini : https://plus.google.com/+HarrySufehmi/posts/3daG2XMjDEt

Setelah dijalankan selama berhari-hari, masih belum juga selesai query nya. Edan.
Ketika saya cek, ternyata query tersebut sukses mensaturasi bandwidth memory – CPU :O

Jika kita bayangkan berbagai jalur data di dalam sebuah server seperti jalan tol, maka, jalur data antara memory – CPU adalah seperti jalan tol yang paling besar dan jalurnya ada ratusan buah :) dan jalan tol ini sukses dibuat sampai macet oleh query ini :D

Maka kemudian query ini di optimize, seperti dijabarkan di artikel yang di link sebelumnya; dan lalu dijalankan kembali.

Kini query tersebut selesai dijalankan dalam waktu 1 jam saja :)

Jika Anda sudah menonton film The Imitation Game, tentu ingat bagaimana frustasinya Alan Turing ketika mesinnya gagal terus menyelesaikan tugasnya dalam waktu yang disediakan.

Dan bagaimana tercengang ketika query nya mereka optimize – dan kemudian jadi bisa diselesaikan dalam waktu sebentar saja :D

SSD bukan silver bullet. Bahkan SSD pun bisa dibuat overload oleh query yang buruk.
Seorang DBA yang berpengalaman bisa membantu Anda menemukan query-query yang buruk tersebut, dan membuatnya menjadi lebih cepat.

Pastikan bahwa DBA Anda paham prinsip dasar untuk troubleshooting soal performa :

(1) Profiling : ukur, tahu cara mengukur performa, dan menemukan titik-titik kelambatan.

(2) Tuning : berdasarkan data dari hasil profiling, kemudian DBA jadi bisa melakukan tuning ke query yang bermasalah – dan bukannya asal utak-atik setting saja….

(3) Benchmarking : setelah selesai tuning, kemudian bisa melakukan benchmarking, dan bisa menyampaikan berapa besar tambahan performa yang berhasil didapatkan.

Tanpa urutan 3 langkah ini, maka proses tuning yang dilakukan adalah serupa dengan praktek perdukunan :) gaib & mistis.

Semoga bermanfaat.



MySQL : Make Your Query Runs (at least) 16800% Faster. :: One of our data…
MySQL : Make Your Query Runs (at least) 16800% Faster. :: One of our data analyst was asked to analyze a bunch of data from 2 tables. One of it  contained… – Harry Sufehmi – Google+

Post imported by Google+Blog for WordPress.

Fixing Java that cannot be re-installed after being uninstalled

One of our computer,  on 64-bit Windows 7,  being used for Minecraft programming using CanaryMod and Scriptcraft so the coding can be done in JavaScript, having troubles after we did so much modding so we had to uninstall the (32-bit) Java.

The problem was that Java could not be reinstalled because the uninstalling process was not clean so that Java files were still in the registry, made the Java installer thought that Java was already installed.

We have download like ten registry cleaner software, tried 3 or 4 of them but the problem remained. Until we find a very useful ilnk that finally made the installation work.

The trick is to type <pre><code>regedit</code/</pre> on Windows start menu and then find any folder contains Java on the folder under <pre><code>HKEY_LOCAL_MACHINESOFTWARECLASSESINSTALLERPRODUCTS</code></pre>. This is a bit tedious but it is totally worth it :D

After that delete these to folder:

<pre><code>HKEY_LOCAL_MACHINESOFTWAREJavaSoftJava Runtime Environment</code></pre>

<pre><code>HKEY_LOCAL_MACHINESOFTWAREwow6432nodeJavaSoftJava Runtime Environment</code></pre>

and then all will be set and the installation will be done just fine.


MCSA & MCP Harry Sufehmi

:: Di kantor yang lama, saya sempat agak sebal melihat sertifikasi IT sangat didewakan :) beberapa kali beberapa kolega dinilai dari sertifikasi yang dimilikinya. Maka kemudian saya mengatakan kepada manager saya, bahwa saya ingin mengikuti berbagai ujian sertifikasi IT  :)

Dalam waktu 2 bulan, saya lulus 4 ujian sertifikasi :) MCP (Microsoft Certified Professional).  Setelah itu menyusul mendapatkan sertifikasi CNA (Certified NetWare Administrator), sertifikasi MCSA (Microsoft Certified System Administrator) dan terakhir sertifikasi CCNA (Cisco Certified  Network Associate) :)

Namun faktanya tetap personil IT paling jago di kantor saya adalah kawan saya yang hanya lulusan SMA (!) dan tidak punya satupun sertifikasi IT…. :) :)

Simon, dia ini skillnya sudah seperti level dewa. Agak pemalas :D tapi jenius. Dia selalu bisa menemukan solusi paling simple dan mudah, mungkin karena pemalasnya itu ya? :D jadinya tidak mau yang repot, ha ha ha. Dan di saat kepepet, selalu bisa diandalkan. Ketika semua orang sudah mulai panik, dia tetap cengar cengir :) dan tiba-tiba masalahnya sudah selesai semua :)

Jadi jika Anda tidak memiliki sertifikasi IT apapun, tidak usah sedih. Asalkan Anda bisa diandalkan seperti Simon, saya hormat kepada Anda. Salut !

Post imported by Google+Blog for WordPress.

Favourite excerpt for the time being

[Mark]”…Codfish Islands was infested with feral cats. In other words, cats that have returned to the wild.”

[Doug]”I always think that’s an artificial distinction. I think all cats are wild cats. They just act tame if they think they’ll get a saucer of milk out of it…”

— Douglas Adams and Mark Carwardine, Last Chance to See

Digital Collections, Data Visualization, and Accessibility: What to Do? (repost)

[This is another crosspost from the Digital Scholarship Collaborative Sandbox blog from the MSU Libraries. The original blog post can be read there.]

In my earlier post “Digital Collections and Accessibility”, I touched upon the considerations we would need to address when building or creating digital collections (or other things that rely heavily on utilizing images such as data visualization) for public use. Here are the questions I put down in that post:

“Given the ubiquitous nature of digital collections, the goal that these collections would be used as part of scholarly activities, and the library’s mission to disseminate the information as widely as possible, there is one aspect that many of us need to address when we plan for a digitization project: how do people with disabilities access these collections without getting lost? Can they also get the same access and benefit from our collections if they only rely on their screen readers (or refreshable Braille, or any other assistive technology)? Can people move around our website easily using just a keyboard (for those with hand-coordination difficulty who cannot use a mouse)?”

So: planning. Planning is an important part when incorporating accessibility into building a collection. Typically, building a digital collection starts with designing the metadata (PDF) and then proceeds to further development activities such as database design, content creation, data entry, and coding/front end development. Whichever process that we develop, we would like to see that the website is well designed and the information presented is useful for our audience (I am assuming that most digital collections created and made available are designed for web access, with an added bonus if they also employ a responsive design.)

Image Display

If you visit digital collections developed by various institutions, you’ll see that they present their collections differently. Many would display the collections be like a catalog that shows an image, the physical description, and related information such as the owner, creator, and copyright statement at the minimum.) Some also include an interpretation of the object (think the label of an object or painting displayed in a museum.)

Regardless how the object is presented (by description or interpretation), accessibility considerations are still the same. The most common considerations: the web page needs to be properly structured by using proper headings; the flow of information presented on the page needa to make sense for screen reader users or keyboard-only users; search forms need to be properly labeled; images need to have alternative text (usually referred to as “alt-text”.) This is when the planning for the page design and coding becomes important.

Consider this page:

Feeding of America page featuring a butter mold

and consider how the flow of information would be read by a screen reader and how a screen reader user might hear it:

Textual representation of the Feeding of America page

Typical screen readers read the information displayed as if the CSS is disabled; they read web content in the order that it appears in the code.
(Bonus: if you have not seen or heard how screen reader users interact with a website, you can view the recording of accessibility test of our e-resources page (.mp4) done by my blind student. We did this as part of our accessibility test routines for the library electronic resources.)

Both images above should be sufficient to give us ideas of how a sighted user might interact with the page and how a screen reader users might hear it. Our eyes can focus on and narrow down to a certain section faster while screen reader users need to listen to the whole thing first before they can work on distinguishing the part that provides the actual information of the object being displayed. Hence, careful planning when designing the metadata and the page is needed to make sure our collection is both useful and usable for our audience regardless how they access it.

Data visualization

A lot of data visualization rely on colored graphics when conveying the information. It is trickier to tackle because of the colors used and, unlike most images used in digital collections, data visualization conveys very rich information.

Consider this example with three different color representations:

Data visualization on children age 0-14 years old population of the world
(Data visualization of world population of children age 0-14 years old. The information is grouped by regions (South Asia, East Asia, Africa, South America, Middle East, Europe and Russia, and North America.) Original data can be found at http://datatopics.worldbank.org/hnp/popestimates.)

By looking at the colors used on the image above, we can see that the information is grouped based on the region (South Asia, East Asia and Pacific, Africa, Europe, etc.) and the color density of each individual block reflects the population density of the area.

Data visualization as seen by a color blind person with protanopia (red green color blindness)
(Data visualization of world population ages 0-14 years old as seen by a person with red green color blindness, such as protanopia.)

The second image shows how the visualization might be seen by those with the red green color blindness (protanopia), one of the most common types of color blindness. Here, East Asian and African regions are no longer distinguishable. Similarly, South American,Russian, and European regions are also no longer distinguishable.

Data visualization on population of the world as seen by a color blind person with achromatopsia (total color blindness)
(Data visualization of world population of children age 0-14 years old as seen by a person with total color blindness (chromatopsia).)

This last image shows how those colors don’t really convey the grouping of the regions to those with total color blindness (achromatopsia, which is a rare condition but still exists.)

The point of these examples: do not use color alone to convey meaning.

As far as I know, there is no practical solution yet for making data visualization fully accessible. Several options that can help increasing the accessibility: supplement the color with text or provide summaries or text description right after the image (alt-text or image caption). If the description is too long to be listed on the same page, create a separate page and link to it. Similar to designing for digital collection, designing for visualization also needs careful planning.

Conclusion

Designing for accessibility for our digital collection or data visualization should be done as part of the planning phase. This would allow us to optimize the output of our work and eliminate or reduce the need to revisit the design for corrections later on. Careful planning on how we want to display the information and to convey the meaning of the graphics/images would benefit all of our users regardless how they access our collections.

Resources

Indigo Apprentice Awards

Sejak akhir dekade lalu, tim kami memprakarsai beberapa aktivitas Telkom untuk pengembangan karya kreasi digital nasional, di bawah platform program Indigo. Salah satu program awal saat itu adalah Indigo Fellowship. Program ini berevolusi, terus dilengkapi dengan komitmen-komitmen pengembangan produk dan bisnis startup digital, dan kini dinamai Indigo Incubator (URL: s.id/4sm). Fokusnya pada pemilihan produk kreatif, untuk dijajagi potensi pengembangan produk dan pasarnya, dan kemudian untuk dikembangkan bersama secara intensif di ruang kerja bersama, termasuk Bandung Digital Valley dan Jogja Digital Valley.

C2C Concept

Sejak akhir 2014, Telkom lebih memperkuat lagi komitmennya dengan mengubah struktur Divisi Konvergensi Solusi (DSC) menjadi Divisi Bisnis Digital (DDB). Salah satu tugasnya adalah menjadi akselerator bagi bisnis-bisnis digital nasional, agar memiliki kekuatan untuk persaingan global. Misi jangka panjangnya tentu adalah menumbuhkan ekonomi berbasiskan kreativitas dan budaya digital, serta mengurangi ketergantungan pada produk digital impor.

Sebagai akselerator, DDB menyusun kerangka kerja yang disebut Indigo Accelerator. (URL: IndigoAccelerator.com) Berbeda dengan Indigo Incubator, akselerator ini menerima produk dan bisnis digital dari para startup yang sudah memiliki traction, dalam bentuk revenue, customer base, atau user engagement lain, yang memiliki potensi untuk bertumbuh, walaupun mungkin saat ini nilainya belum significant. Dalam akselerator ini, digunakan fasilitas bersama untuk memperkuat pengembangan produk dan bisnis; termasuk penggunaan common service enabler, marketing channel, dan sinergi pengembangan lain. Indigo Accelerator bersarang di Jakarta Digital Valley (@Jakdiva) di Menara Multimedia, Jakarta.

Startup Accelerator Facitilites

Pintu masuk ke akselerator ini dapat berupa keluaran dari Indigo Incubator, dari bisnis digital yang diajukan proposalnya ke Indigo Accelerator, atau dari bisnis yang dipilih tim Riset dari ekosistem digital di Indonesia. Namun agar proses ini lebih cepat dan terbuka, DDB mulai menyelenggarakan Indigo Apprentice Awards (URL: s.id/4bE). Ini adalah kompetisi yang terbuka bagi para startup Indonesia, untuk diakui sebagai bisnis yang memiliki potensi pengembangan terbaik, dan untuk memperoleh fasilitas pengembangan bisnis dari Indigo Accelerator.

Indigo Apprentice Awards

Juri untuk Indigo Apprentice Awards, selain dari Telkom, diambil juga dari komunitas-komunitas yang merupakan stakeholder pengembangan startup nasional, semisal Startup Lokal dan Founder Institute.

Founder Institute Jakarta

Pendaftaran dan informasi lain tentang Indigo Apprentice Awards bisa diperoleh dari site berikut: http://s.id/4bE atau via account Twitter @ddbAccelerator.